Of course it has to be that way, since this is a fantasy for people to jerk off to it. No one fantasizes with an ugly guy in a smut scene. This is a fantasy about people pretending a handsome guy does this stuff to them, it's about pleasure, not really about true principles or values in life.
Drugged and rape by handsome=okay. Drugged and raped by ugly=not okay. Alright then...