Martial Arts?

I'm just a few chapters in and there's not much to rag on, but even though I don't hate the fact that he trains some sort of self defense means (though Rice Cake seems to be able to guard him just fine in my opinion), learning Swordsmenship and Beastmen Martial Arts both seem to kind of harshly wear down his hands. From holding the wooden sword and swinging it, to somehow crawling around on all fours through all-terrain. The hands of a chef are as holy as that of a violinist or pianist. If his hands numb through this sort of coarse exercise, that would be no good at all. Not a good combination.
I'm not saying it's impossible, I'm saying it's kind of late in the game. If he had some fundament, it would be easier for his body to get used to it - like, had he started cooking and training his body at the same time, for instance. But now this is just going to make his hands, which where solely trained for work in the kitchen, do something so straneous, which is going to be detrimental when cooking later.