gath

Because it’s heavily fetishized and written by straight woman for straight woman

You mean movies like “love, Simon” and whatnot?

or actual yaoi?

It’s could be the same reason as to why people don’t often talk about “being gay” or just gay people in general. It’s different and people don’t understand it, well there are also people who are trying to close the gap by spreading awareness and explaining it’s not that different from a straight relationship. But for the genre Yaoi, it’s mostly known for showcasing a lot of the “explicit” parts of a relationship. Usually, stories or with straight couples revolve more around the plot and characters, not what Fujoshis and Fudanshis often look for in Yaoi. So, many people tend to categorize Yaoi with Hentai...and I guess people don’t just casually talk about hentai? If you were to say, Shounen Ai, I’m sure there wouldn’t be much to feel ashamed about.

I don't think it is gross, nor do my friends. I think the reason why there are more people ashamed of liking yaoi is because there is generally alot of sex scenes, I would feel uncomfortable talking about even hetrosexual sex so I avoid mentioning I like yaoi that much unless with close friends as it is pratically talking about porn/hentai. But shounen ai is different, as it is a BxB but without any smut, I always talk about that with friends who share common interests. I have one or two friends who do find it uncomfortable to talk about romance between two males but they always keep respectful.
It is sad someone can feel ashamed for liking a genre, perhaps it is because it is created by straight women, as another comment said, and does really stereotype gay people sometimes, like the terms "seme" and "uke" are quite rare in real life yet so very popular in manga. I am a proud fujoshi but there seems to be a common misconception, that we fujoshi/fundashi see gay men as nothing more than sex toys for our imagination, that is a really disgusting misconception to be honest. Also, maybe it is because themes like rape, incest, threesomes, etc. are much more common in yaoi then hetrosexual mangas? I am just guessing as to reasons why yaoi may be so 'infamous' so to speak.

I don't think mainstream media should be compared to yaoi. That's like asking why people are fine with violence in action films but are turned off by torture porn. Maybe.