To me #2 makes so much sense and fits very well with the time period to say of what the manhwa seems to take place
In regards to #1: Due to the type of setting it would be natural for people to asume she's a young boy tho. As females during this type of time (if this story setting goes by our world's standards that is) have always had long hair and from what I know it was not acceptable for them to have short hair (unless braided to the back of the head or such). Hence why, due to her androgynous looks and lack of overly visible chest, she's seen as a boy quite easily.
I do think her feminine mannerisms should throw some clues (her general behavior isn't exactly boy-ish nor is the way she carries herself, imo), but I don't think servants and whatnot would care or have time to pay attention to that anyway. I doubt it's because they are conveniently dumb. It might just be a lack of interest, especially since the person in question is of lower status than them. That usually yields a none-caring attitude in many regards.
3 possible answers if you want them. Just my thoughts.
1. In these types of stories, minor characters are generally not that smart for the authors convenience. The rumors say she a boy and why would a rumor like that exist if it isn't true? They expect to see a boy so they see one.
2. They think that she is a younger boy. In the time period/setting it isn't uncommon for older nobles to keep young boys around as a "hobby." Even the Archduke thought she was like 15yrs old. At that age it is entirely possible for a boy to still have a high voice and a boyish/neutral frame. Despite what we imagine in our heads, we don't know what Chloe's voice is meant to sound like. I imagine her voice as very high pitch, but it could be lower and more neutral sounding. Also, she said herself that she was boy. In the public's mind, what reason would a young girl have to lie about that? She dresses like a boy and her loose fitting clothes make it easy to miss her breast as they are small. The tailor asked if she was a boy or a girl in the first place because of her loose boyish clothes. He, and many others, originally thought that she was girl but asked anyway.
3. Their prejudice ideas of homosexual relationships rationalize that it makes sense that he would have feminine traits/mannerisms. It's a "hobby," to them (manhwa's words not mine). Loose clothes hide her breasts and her voice is unknown to us. The author mentioned the prejudice a couple of times.
This translator used "fetish," but the usual ones used "hobby" which to me fits better.