One has to understand that the Valley of Geniuses, as a singular example, was responsible for maintaining the souls of often long dead or transcended geniuses alive, and also supplying them with enough power to pass on their legacies. For a tower that's been functioning for millions of years, the amount of energy that requires is astronomical.
Simply put, celestials would be able to make a mockery of the system, even to the point where they could forcibly take legacies that weren't meant for them. If you were a celestial? Go fight it out in the Valley of Geniuses on your floor. It was as simple as that.
Of course, since the tower was capable of having the energy to support another level of the Valley, energy constraints weren't the main focus behind its reasoning. Rather, it was to allow other geniuses the chance to grow.
