It only took a few hours for the group to travel from their point of arrival in Ganaway Refuge at the maximum speed that Elpam could take the group from Minos to their goal in the center of the island.
Elpam tried to talk to the group in the middle of the journey to relieve his own tension and that of his companions, but also to appear friendly to these strangers and gain a better understanding of their origins. They obviously didn't know that Minos and the three human women had been there a few times in the past.
Hundreds of millions of creatures came and went from the island every decade. There was no way the local leaders could have known about non-Supremes passing through the area.