Considering this explicitly unique situation, to have an upper advantage against the Demons two steps have to be taken.
Although a good portion of the Town could have done the same if they paid attention, they didn't. It is Dilli Flurry who has made the firmest theory of where the Portal might be, by simply using Demonic traces.
And of course, they did not just burst in the Castle after forming a theory, that would be a bad idea in many aspects.
For example the Town could be caught off Guard if Hundreds of Demons suddenly decided to appear everywhere, even people as physically powerful as the Calidum Lutum could suffer large losses from a low tone Ambush.
Two but equally important, if Laza, Kazer and Dilli just went into the Castle without informing anyone. If the Demons actually overwhelm the three Adventurers, there will be no saving them, no back up.