Sam did not decide to tell them about his adventure in Terraria. Well, he did not want to make them more worried. If his family gets to know about those Abyss humans, dark humans, and those other dangerous things about Terraria then they will not let him go back to that planet.
On the other hand, Gloria, Hina, and Victor are currently digesting all the information that Sam has given them. Even though he did not give them the full details, all of it is still very serious information which naturally takes them some time to digest.
Who could have thought Sam would find something like this inside a basic Dungeon? Also, after hearing all the explanations they became sure not just only inside that Dungeon, but probably they would be able to find that kind of teleportation symbol inside the Dungeon.