Even though the angels started to retreat, they already had many of their members in the Underworld and Realm of Gods. That said, they took several months to send everyone back, leaving the Temporal Path Behind. Finally, the Devils got a chance to organize in the Underworld and attack the Temporal Path Entrance.
The Angels were in very low numbers already after the majority left. The Archangels themselves didn't see a reason to continue there to guide the last bits of their forces, so they were long gone too. With that, getting rid of the Angels that were left behind was easy, and the Devils soon retook the Temporal Path Entrance.
