Although the subordinates of the White Tower were all eager to eliminate Jeremy and his companions as quickly as possible, each of them had different motives.
Every one of them wanted to claim the greatest glory for themselves, making cooperation among them completely impossible.
The three primary lieutenants of the White Tower had all formulated their own secret plans, and they directed their respective forces to act according to these personal agendas.
As a result, Jeremy and his group didn't have to worry too much, at least in the short term.
The White Tower itself lacked the resources to focus on Jeremy, and the subordinates who did have the energy to go after him were incapable of working together.
This gave Jeremy enough time to continue exploring the Tower of Exile undisturbed.
At the very least, he could proceed through a few more floors of the Tower without any immediate threats.