Recently, he had looked back upon his past transmigrations. If potential points were the energy to be used as consumption for his body, then the experiences he needed to amass to form a Soul Seed were the potential points he used to upgrade his body to the limit, which were converted to energy for his soul and stored. This was a gathering process that was extreme to the limit, but this was also the only way for him to reach a higher realm.
There were two more important reasons as to why he did not immediately boost his attributes.
If he activated the Fifth Star and reached the highest limit for his attributes, then he would need to consume a large amount of life force. The little bit of vigor that Slaughtering Hands provided were nowhere enough. Once his life force was damaged, he could achieve a similar effect as restoration through supplementing his vitality with potential points.