True, both are hurting. This last update showed the present Daiki remembering how Sabato reacted after everything - he now regrets hurting him and the way he acted about losing his memories...
This is why people are mad at him though: http://www.mangago.zone/read-manga/kachou_fuugetsu/tr/v04/c024/pg-16/
He didn't care about how Sabato felt, and he purposefully didn't tell him that he had no memory of their time together.
I do hope that he regains his memories, but there's a chance that he won't (happened in real life, a woman lost 5 years and didn't remember her husband (they made a movie about it), while things eventually got better, she never regained those 5 years). If he doesn't remember, he's really going to have to work hard to get Sabato back.
I have the feeling that the two are thinking at cross purposes, however. I believe Daiki thinks it's only the fact that he lost his memories and raped him (while that would often be enough on it's own I certainly do agree) that hurt Sabato. But that what really hurt Sabato was the fact that he dismissed their relationship so casually and didn't treat their time together as preciously as he wanted Daiki to do.
Awe that movie I watched it. It was really moving how the husband stuck with her and tried to help her the best he could, even when she was confused, or she didn't trust him. Even if she never remembered.
Many people hate Daiki right now but it's clearly been conveyed in this chapter update that both are hurting. And so are we.