The expressions Chu Mo and Lan She gave to him and Han Ya made An Yuan uncomfortable. Though he really doesn't have anyone in the real world to return to, that place is still his world. To remain sane in this world, he needs to have a goal at the very least. It just happens that his chosen goal is that he wanted to return to his own world.
But upon hearing that he wouldn't be able to have Han Ya by his side when he returns to his real world, his attachments to that place slowly faded. He had long chosen to love Han Ya as his lover. Though they haven't done anything as lovers, he knows that Han Ya loves him as much as he does. It was because of this that Han Ya had become the most important person in his life.