Indeed, it was the white-robed swordsman that had a hand in what happened!
Nevertheless, it wasn't cause he didn't want to see these people dying or anything of that sort.
In fact, the more people that die in this temple, the more beneficial it would be for him, which is related to a secret of his that is at the same time related to why he would be in such a weak world.
Meanwhile, after the white-robed man saw Abaddon on top of the mountain, he had to make the temple clear of any other people so the space inside of it wasn't disturbed, and everything went as planned.
As to why he would do such a thing after seeing Abaddon inside his secret land, that's something only he did know.
Oliver, at this moment, couldn't take it anymore and ordered everyone to head back to the clan; even the dragon slayers were no exception.
