Selling herbs was a deed performed by poor villagers, who made their living by risking their lives to gather herbs and magical mushrooms and stuff from the spiritual forest and the other wild habitats nearby the Cloud City.
The lady assumed that he, too, was one such herb gatherer.
As for why he was able to dress himself so well, they believed that he must've picked up and sold a good herb for many gold taels and used his recently earned fortune to buy luxurious clothes and accessories, which also meant that he had no gold left on him and that he was no better than a nicely dressed poor hillbilly who had to risk his life daily to gather herbs and sell them to make a living in the near future.
No matter how handsome, a poor person was not worthy of them.
A poor person, who had to risk his life every now and then, was even less worthy of them than pigs dreaming of eating swan meat.
They dared not take an interest in a man with a high life-risking job.
