man what has he done since transmigrating for people to still gossip about him

hasn't he been rather quite and not caused any serious trouble to other people in the (i think?) 5 years since transmigrating? cause it seemed like he pretty much kept to himself except for social occasions that he couldn't back out from? yet people still talking wild rumors about him doing crazy stuff? like were did the harassing a lady while drunk and nearly beating a man to death come from? the only thing I can think about is it twisted version of the lie he sold to his family's personal doctor about his injuries being from a drunken brawl? but no one should of known of that, they only showed us the only other person he met while still injured was the prince so i don't think anyone outside his house and the prince should know (and the prince knew that was a lie). MAYBE some maids from their house spread the gossip about it but even then it got way more twisted down the rumor pipeline