Javier's worries came from the fact that his Penthouse was empty. Ever since George's mother gave the Pethouse to him, he never stayed there once. So naturally, the wardrobe was empty, there was nothing on the fridge, and there were no belongings of him that was inside the house. There were no signs that he lived there.
If it was only that, there's nothing to care about.
However, George and his parents were different.
Javier was worried that his brother might take it in the wrong way. It was a gift from their dead mother, so not using it might hurt his feelings.
George and his parents were sensitive when it come to these things.
Whenever he refused their endowments or left the things they gave him untouched, they tend to overthink. His restraint to take advantage of them was seen as him treating them as strangers.
