Although they had recently talked about going with the Marquis Valoide if the conversation with Patricia and her friends failed, Andrew was reluctant to do so, because he knew that the Marquis would take advantage of their situation, which is why Andrew did not want to talk to him.
While it is true that after the war the marquis and his wife had tried to repair their relationship, Andrew knew that at this point there was no way to return to the relationship they had before.
Candice and Teresa might be friends, but Teresa had made her position as a noble of the empire clear, and Andrew realized that the Marquis would never trust Candice as he once did.
