Master Sun didn't take out the Command Token immediately after stepping onto the Road of Despair. Instead, he relied on his own abilities and walked several miles along the road.
The temporal chaos layers were like invisible Flying Swords, rushing at him amidst the cold wind. With each encounter, they threatened to trap Master Sun. Even the lowest level of chaos layers that could catch only the Three Mountains Divine Artifacts would limit his strong divine consciousness to a range of only ten meters, making it virtually impossible to find the path back.
For the Red Clan barbarians' believers, who did not cultivate their divine consciousness, falling into these chaos layers would mean certain death. Thus, the road was indeed one of absolute "despair" for the Red Clan barbarians.