I think it’s more like the MC’s father married the brother’s mother...so the 2 brothers are in fact stepbrothers with no blood relation and Natsume isn’t related by blood to the father either... it’s the only option that would make sense from my point of view...
i feel like the bro is his cousin or step brother and the dad an uncle