I was thinking the similar things. :))
In my opinion I think Siwon had/has a lot of insecurities. It was hinted his family fell on hard times so he probably didn't feel worthy of being nice to Byul anymore. Not to mention his crush on him, and Byul being another alpha makes it impossible for them in his eyes to be together given the omegaverse concept. And so he most likely drifted from him on purpose. Then moved away.
It's why he was trying to keep his distance from Byul since he transferred back to the same school as him. He likes Byul so much but thinks he doesn't have a chance given their second nature genders and the social gap between them.
I wonder if Siwon had a bad experience with kissing, could be with Byul who has forgotten, somehow. He for some reason doesn't remember much when they were younger and it lead to a misunderstanding with Siwon, who thins he was avoiding him. Who knows