The reason why Andrew and the others had not noticed the hooded man watching them was not because they had been careless, but because this man was very skilled in stealth techniques.
He was also helped by the fact that Oliver, Zany, and Andrew had not used all their power to check the surroundings, as doing so could be seen as an act of hostility towards the city's experts.
That was an unwritten rule of almost every city on the continent. No matter how small a city is, there will always be at least one expert in it and this is why this unwritten rule took hold.
Andrew and the others took their horses unaware that they were being watched and rode towards the next city. Their destination was the capital, but it was a week's ride away and they had to pass through two other large cities before arriving.
