After the small commotion surrounding Toz's inclination growing died down, it was about time for Toz to begin heading out. He had nothing left to do, and it would be better to leave Yellow Sparks as a part of the still ongoing evacuation instead of waiting for the chaos that would follow Hugh bringing the Rat King's dead body back. If he left some time before Hugh could bring back the Rat King's body, there would be less suspicion around his involvement even if something else points in his direction.
There were also very few people who knew Toz and that he was still on First Ignition. Amber, who had frequently talked to him about everything regarding the rat extermination progress, was, of course, aware of what Toz was doing. But as long as he told Amber that he had noticed the Rat King's power from a distance and decided it was out of his reach and that he would evacuate, there wouldn't be any rising suspicions from her end.