I BELIEVE (if I understood it correctly) that they (or the little girl Narumi, at least) identified him as yokai or part-yokai because the eye that was covered by the patch was possessed by a little yokai-demon something that was guiding him to the tree (or the treasure or the remains of the guy he knew, I can't remember which exactly, now). It then left his eye, so he's human as usual, but they either didn't know the difference (because they can't see everything) or just still identified him with yokai-ish-ness anyway.
Alternatively, we have heard through the story of other humans who become...more than human if they spend time with youkai. Perhaps what happened to him. It was beautiful :)
But quick question: I thought Ijima Ritsu was 100% human, but now youkai and people are saying he's not. Is he changing? Or something?