The two elders weren't trying to scare them. Although the labyrinth city wasn't big, it wasn't far from the tyrant's blade school's headquarters. The underground labyrinth next to the city was huge. No one knew how big the underground palace was. If the ghost cultivators really wanted to build a base there, it would have a great impact on the tyrant's blade school. Most importantly, the tyrant's blade school could find the base because the underground labyrinth was too big. Even if the tyrant's blade school sent out all their cultivators, they might not be able to find the ghost cultivator's base.
If they couldn't find the ghost cultivators 'base and allowed them to slowly develop in the underground maze, it would undoubtedly be like placing a time bomb next to the tyrannical blade sect's headquarters. That would be too dangerous.
