Contrary to everyone's expectation, Adrian doesn't seem to be convinced by the fact that it's Irish who's behind the reassignment.
In the past, he used to let his family members do this kind of things. Since he's working compulsively hard and for long hours every single day, he's rarely at home. Because of that, Cassidy was all by herself most of the time.
His mother and sisters would often visit their mansion and help his incompetent wife in managing the household. Considering that she never excelled both at their family's business and domestic works, they would offer to teach her while he's away and busy.
What Adrian did not know was that these women were being very brutal while they were doing it. All of the witnesses and houseworkers were being silenced by either Gertrude or Irish every time. That brutality never reached his ears.