Exactly my thoughts
While this is 100% true, I think the whole idea behind it is that he believes he has the right to know what type of relationships his biological child's current caregiver keeps around. Since in a way it would affect his child in some degree.
Which when you think about it kind makes a lot of sense.
I once heard a story of a woman who's baby died because when she left it at her mother's place for the day her mother's boyfriend (whom the baby's mother didn't even know about) had made contact with the baby while sick and the illness killed the baby. In the end of the day the child's well being is what comes before EVERYTHING.
LITERALLY
I don't understand how the bio dad was so pressed when they didn't tell him they were together like how were they supposed to know you have gay friends or you're not homophobic? Their PRIVATE relationship doesn't need to be disclosed to you. If Seiya said Ibuki was the damn babysitter, he is unless they tell you otherwise