
They make it clear in the story that the guy has this special ability to communicate and understand the cats. At first I didn't understand why something like that was mentioned so early on in the story. But now it makes sense that it was mentioned, because if the cats were actually people with only some cat charateristics than everyone would be able to communicate with them and not just one person. This guy was likely given the task to take care of the cats by the crown prince because of his special ability. The crown prince probably really loves cats and wanted someone who can communicate with them to take good care of the cats. Other people are unable to communicate with the cats, which is why at the end the blonde guy, that showed up, says that it sounds like that guy was just talking to himself.
My dumb ass only realized at the end that everyone else sees the 'catpeople' as actual cats because they are, in fact... cats. And the artist only drew them to look like people with cat characteristics so it's easier for the reader to enjoy the story