When Rowan had swept this weapon away from the Undying and sent it hurtling into the horizon, it was already traveling at many times the speed of light, and if Berrion had reacted immediately and say, froze Space/Time, maybe using his higher dimensional powers to bring back the memory of the weapon, he might have been able to stop it, but Berrion was still restricted by the power of the Primordial Weapon.
The same reason Rowan's Time Stack ability could not work on his Bloodline of Sheol to multiply his Soul cultivation, a Primordial weapon could not be manipulated by higher dimensional forces. It was simply too dense.
You could not freeze it in space/time or pluck it from a Memory, it could only be carried by those with great strength and mental fortitude. This was what made Astrolabe so valuable to Rowan at that moment, it only controlled pure speed and with the upgrades it had gotten, it could carry the Primordial Weapon.
