In the audience, only a few people who were free or had their immediate wards participating stayed while the busy folks left.
Naturally, the ruling class of all three races left, unwilling to leave their domains unprotected for three long days.
Even if the elves could dismiss it and stay, they had no reason to, as the test was really long, and a few messengers were left in place.
For example, if someone stumbled upon the location before the third day, the ruling class would be alerted by their remaining servants.
Of course, the Academy wasn't formed by a bunch of idiots who'd allow such an incident to tarnish their prestige.
That's why the final location was nothing short of a lie that would only show up on the third day, preventing anyone from finding it.
Obviously, the ones who graduated from Solarnelle Academy were long aware of their methods to get their decided results.