Tang Wanshan usually lived in the Medicine King Valley with some servants. As for his son, grandson, and the others, they were basically all outside because they had many businesses in the Tang family. Tang Wanshan's son, who was Tang Qianqian and Tang Yi's father, ran these businesses outside and only came back to the Medicine King Valley during the holidays.
As for the people outside the Medicine King Valley who begged him to treat their illnesses and save their lives, Tang Wanli was already used to it and basically ignored them.
If he had to save someone, he might as well die.
It was normal for people to die of old age and illness. Even if he was regarded as the Medicine King, he was not a god and not anyone could save him.