The environment within the confines of the Imperial Palace was specially created to ward off individuals who had hostile intentions. The downside was that the victims weren't chosen.
Every single visitor of the Imperial Palace was subjected to a sort of mental and physical torture. The sole remedy was to get accustomed to it. Meaning, one had to visit the Imperial Palace repeatedly.
Although the impact on Sorcerers was worse, non-Sorcerers would not be spared.
Having said that, Gover knew exactly what Vhelian was going through. It came as a shock to him as well when he entered the Imperial Palace to acquire his Noble Title. He was still having a hard time as of the moment, but not as bad as the first time he felt the pressure.