I think they're assuming he might be gay or just has an unhealthy attachment to Joe since she was the only one to stand by Kyle. I think in peoples mind, the majority just thinks he's attached bc of Joe being his loyal retainer and also the goddess' messenger
Does the court knows that Joe is a girl already or are they assuming that their new emperor is gay?