The truth was that Xiang Weimin had no answer to that question. It was true that neither Grandfather Xiang nor Mu Chenyan never came between the siblings. They might have stopped Xiang Hulin from reaching out to his daughter because of that promise but when it comes to Xiang Weimin and Xiang Wai, they didn't have the same concerns.
They never wanted to come between the siblings. That was never their intention. However, it was Xiang Weimin and Xiang Wai who chose to distance themselves.
As for why? It had mostly to do with the fact that they were both young and they both took their parents' divorce a bit hard. But seeing how even their grandfather chose their youngest sister, both of them were a bit resentful. In fact, they both resented Raelle at one point.
They were kids and they wanted to put the blame on something. They chose to blame whoever they heard about. Xiang Weimin didn't even want to mention in front of Raelle what he had heard when he was young.
