Kagura Hikaru brought Ayakoji Chiyoko to the agreed place.
Under a maple tree in the courtyard.
....What are you thinking? That he would take Ayakoji Chiyoko to the rooftop of the old school building?
No, no, no.
Imagine this scenario.
You are picked up by a woman who likes you, and you like her too. You quickly fall in love, and after doing everything together, you learn that she has a partner and you are the third party.
You angrily leave that woman and find another whom you can truly trust.
Then one day, the former woman finds you, tells you she has separated from her original partner, and wants to get back together with you. Would you bring her to meet your current lover?
The reason why Kagura Hikaru didn't take Ayakoji Chiyoko to that rooftop is somewhat similar.
But seeing her looking up at the blooming maple tree with a hint of joy on her face, he couldn't help feeling guilty about his own stinginess.