If I remember correctly, he's mad at himself for not asking what's the reason she said she cant get pregnant like is she sick? Why did he not asked further about her condition or why he took it lightly something like that. I don't think this is where the matter of child was mentioned yet but maybe in the next chapter
Wait a minute is it me or didn't you'll read in the starting that Lucia always got dreams about her future so to avoid that she married ou ML and for the menstruation thing. She knows the duke didn't like children so she ate some plants from her garden to stop her menstruation. From the same future dreams, she knew about the plants and she even knows how to cure her menstruation thing (╯°Д °)╯╧╧
She did that when she was younger because she thought menstruation was an illness and ate a bunch of those herbs and this was before she got dreams so that's why she's saying she can't have children. Also about my first comment I thought this was one of those timeline manga, forgot she had dreams instead.
I don't know quite how to put into words my understanding of his angry but I'll try lol. I feel that his anger is towards whom ever was in a sexual relationship with her before him for her to know and be so sure about not being able to get pregnant. Like, its not something you instinctively know, you have to go through trial and error with a man before you are THAT confidant to firmly say that you can not get pregnant. And the duke just has not realized the source of his anger yet.