When Xavier finished perusing Castiel's memories, he threw him on the ground. The warlock had an inflated sense of self-worth, but he did not understand he had been used by unscrupulous witches.
But he was not innocent.
According to the memories, Castiel was bound to the mirror by the witch after swearing the oppressive oath to sell his soul and capture other spirits for the mirror. He was sceptical at first, but he was willing to take the risk.
The first victim of the mirror after the initial acquisition was Castiel's sister. At the time, Castiel did not have a job and was living with his sister who worked various small jobs to support him.
She never thought of abandoning him since their parents died, and he fell in with bad crowds that ended up in bad injuries. She paid for his hospital fees and continued covering for his rent and food.
However, she was only human and under a lot of stress.