Fallen Angels dwell in The Abyss, holding royal status with formidable strength; no other royal beings aside from demons would willingly provoke them—the tribe that betrayed the gods.
As for Fallen Heroic Spirits, the majority reside in the Human World, and there might also be Fallen Heroic Spirits in The Abyss.
The reasons behind a Heroic Spirit's fall are numerous...
Such as falling in love with a Spirit Summoner only to discover that your beloved is a scumbag two-timer.
Or the Spirit Summoner turning to darkness, and the accompanying Guardian Heroic Spirit turning as well.
Beyond these two, there are various reasons; the number of Heroic Spirits in the Divine Realm isn't fixed forever.
Heroic Spirits can disappear, die, or be reborn for certain reasons, and as for Fallen Heroic Spirits... it seems like the Divine Realm gives Fallen Heroic Spirits a chance to mend their ways. Those who reform are welcomed back by the Divine Realm.