Cant recall if the whole parents story was ever elaborated but I hope the parents have divorced before the dad fell in love wt tarou-san cos if not then they really deserve to be hated by the brothers.
But if the brothers are actually just putting blame on tarou for the divorce of their parents (in case the divorce happened because they just fell out of love & no other party involved), then that’s actually sad for tarou cos he did nothing wrong & just happened to fall in love with a divorcee with kids