The first was probably the simplest. It was a group that did not believe in Savine's capabilities of digging out a seed from inside the Divine Tree. They happily agreed to let Savine 'try', just so they could see her fail.
The second group was one that did not wish to allow her anywhere near the Divine Tree. There were traditionalists among them who did not wish to see the Tree desecrated, and there were nobles who feared deeply the consequences of allowing Fest to possess its own source of Vasas.
With how quickly the duchy was growing even without the support of magic, they did not dare to even imagine what they could achieve with access to mana.