What did the ML’s father do to his mother?

ML keeps saying that “he is like his bastard father”, so I assume that his dad did nothing as his wife was getting ridiculed (she kept yelling that He family doesn’t deserve love) and treated her coldly (considering his reaction to her emotional breakdown and how his father is also sometimes in denial, forgetting how his wife is dead for many years. just like ML when MC ‘died’).

But everything is just my assumption from context clues.