I think simply the fact that you are worrying about this already shows that you are a reasonable person that can clearly see the difference between fiction and reality. Scientists have already done work on similar topics (maybe you have heard of the "Do shooting games make real shootings more likely?"-debate). As far as I'm informed we are quite ......
Yaoi itself isn't problematic. The delivery is. Alot of yaoi romantacize rape and abuse. They normalize toxic behavior in a relationship which isn't OK. If you don't go around gay people irl with a note book and ask them to fuck infront of you, I don't see how it would be fetichizing gays. It's fiction and most of the time yaoi are in no way, shape......
Somebody you really hate slaps you in the face, What do you do?