Turns out, as long as you had permission from the dark elves, you could basically come and go in the forest without worrying about anything and that included flying.
But the dark elves didn't just hand over permission that easily. If that was true then the people of this country wouldn't have had to carry around those magic balls just to get around the forest.
Apparently, there were some deep secrets within this forest and that's why the elves kept all the secret places, well, secret.