Vilgefortz was ambitious and cunning. He hid his true feelings even in front of Emhyr, a pawn on his chessboard. He learned about Olivia's talents from Emhyr. As arrogant as he was, he acknowledged Olivia as a legitimate threat. That little girl could be trouble if left unchecked. Even so, intelligence wasn't enough in this world. When faced with brute force, intelligence could only do so much.
Nevertheless, he agreed to Emhyr's requests for the sake of maintaining their good relationship.
Vilgefortz was incredibly strong already, but he had not yet had enough. Of course, he always kept a wild card under his sleeves. Mind games did not always help with brute force; he still needed to fend off the arrows that came from behind.