
Ok I won't spoil much but basically after he remembers he was in constant battle between hating her and loving her like he did now. When remembered, his memories about her lost a bit and he thought she's lying to him all the time (he pretends like he doesn't remember). For the most part he was very mad and triggered that he was lied to. His personality was kind of violent and volatile.

I mean....Kind of a bit complicated. It's not about him being abusive or not. It's more of him being kind of forceful compared to how gentle he was when doesn't remember, to Lee Yeon. But Lee Yeon was not suffering or anything (except on the last bit of that arc). She was just taken aback by how different he was since she doesn't feel the same warmth anymore.
Another comedic misunderstanding no wonder he was so damn triggered when he remembers lol