I reread this again and I was at ch 25 and he said “no, not at all” what what does it ...

I don't think it has to do with his feelings for Haebeom. I'm pretty sure when he says "not at all" he just means he isn't sure if it's the best option to raise his child in a country that isn't very accepting of gay people. And he is unsure of whether or not he should keep living there with Haebeom or leave.