Julia understood that Arabella's marriage could have been ruined if she was seen with Andrew in that state and Ferdinand misunderstood.
Sera and Andrew's marriage could have been ruined as well. They could have even lost their child if something happened to Sera due to all the emotions she could have felt if she saw Andrew looking all flushed and aroused while with Arabella.
And if the other princesses and ladies had seen Arabella and Andrew in such a state, and Ferdinand and Sera were both emotional, all sorts of rumors would spread.
Arabella and Andrew would both be seen as cheaters, who, despite being married, cannot even control their urges for another's spouse, during such important times when they were supposed to be focusing on the Assembly.
For royals and nobles, concubines were usually allowed to have more children to carry on the bloodline and ensure there would be heirs so the family line and power would not decline.