Found it. Got the affair thing wrong. Noblewoman forced him to do it with her, or else she would accuse him of rape. Also before that he was rescued from being tied up and forced to have sex with village women because they wanted a child from him. Pretty messed up stuff, but the rest of what I remembered was right and his relationship with the present-day guy (who also has a nice body) is really adorable. Muscular guy is a knight, a greybeard and HUGE. It's called Liebling!
There's an evil bastard in it that does horrifying, gag-wothy things to good, but hapless other characters, so DEFINITELY not for everyone. I quit reading when my time demanded other things from me, so I lost track of it. I see it's now completed, Bara, yes. Plot-heavy, yes. Adorable main couple, YES. Sexual violence, unfortunately, yes, but central to the plot
I'm really struggling to remember this one, so my details may be off: big muscular guy, I think was a guard or a knight being horribly punished for having an affair with a married noblewoman (he was innocent), somehow is sent to the future (present day) and ends up being taken home by some nice guy. Both guys thought they were straight, but began lusting after each other and of course they did it. Both end up going back to the guard/knight's time. Obviously, evil sorcery and medieval politics are involved