Damien's heart tensed up when he realized that there must be ley lines all over the places that channeled ethereal energy to power up the teleportation circles located through the base of the Forsaken.
And this leyline must have had a single energy source or medium for effective channeling, which must have been none other than Yohr since he must have built it to gain more ethereal energy on the pretext of channeling energy to these teleportation circles whenever it was needed.
But now that Yohr was dead, there was no longer a medium to channel the ethereal energy. If it was just a problem of energy source, Damien wouldn't be worried since he could simply use his own ethereal energy to power it up.
However, this teleportation formation was not created in a way that would allow him to power it up with his own ethereal energy. The only way was through Yohr. But it was now no different than having the wire of an appliance get cut off from the power socket.