Since it had been proven that the four children had Paxton blood, of course, Benjamin couldn't just abandon them. He put the four children in Red Rosemary because he knew his father inherited this place upon Daniel.
Somehow his mother let her husband inherited this place on her stepson, even though she often tried to take everything Chloe had indirectly. He never could understand why his mother let the beautiful housing go just like that, but he was grateful his mother did.
At that time, when Chloe left this world, his mother also left him two years later. He was still mourning and adapting that he had been left alone, yet four little girls appeared into his life abruptly.
In the beginning, he disliked the drastic change and only accepted them out of pity, but his feelings shifted into affection when he saw the girls grow up. They were adorable and loveable. Not to mention, Catherine's face reminded him of his beloved sister.