Actually, if he was brought into the family when he was young to act as their butler, it’s more likely that he’s anywhere from 3~8 years older than she is. Obviously this type of role where he teaches her is just made up for the manga, but it wasn’t that uncommon for wealthy families to take in children a few years older than their own kids to be trained as butlers/help under the senior butler/help that would retire when the kid gets older. So at the point where he “joined” their family, it’s likely that he was still partially training for his position is only a few to a couple years older than her.
Are the age gaps big?