In the Upper Realm.
There are lots of Gods and Mythical Beings lurking near the closed gates connected to the Mortal Realm. Not only do these gates forbid anyone from just going down, but only those who ascended from the Mortal Realm could also enter. But as long as you ascended, it would be harder to go down anymore as the four Mythical Families held the authority over the gates. With the permission of the four rulers, it would be impossible to do so.
That's why the boundary gates are yearned by almost every human in the Upper Realm, they haven't been able to descend even once back to their realm as the rules prohibit them to. They would always watch the closed gates waiting for an opportunity to jump out as long it was open.