When Grandpa mentioned his marriage to Yan Xiaye, he was quite surprised at first and then easily seduced the reason from Yan Xiaye's lips why she agreed to marry him.
Since then, he seemed to never look at her directly again.
He probably thought that a woman of no worth only deserved a glance and was worlds apart from the gentle and subtle Yan Shuirou, not worth any of his time.
After the wedding, he treated her with his usual indifference, no different from how he treated the furniture in the house, except occasionally, mockingly remembering that this woman was legally his wife and also key to inheriting most of the Li Family's fortune.
Because of this, he found that Yan Xiaye's attitude towards him had never changed, even though he never touched her or explained the changes from before and after their marriage.
He gradually realized Yan Xiaye had never said it, but... she probably loved him.