I prefer this theory more because the duke said that the previous duke was too old and sick to do those stuff. Not to mention, her mother was also young back then and rejected the count's love, which means she is definitely in love with someone else back then. I think that regression was also the reason why she ended up in the orphanage. Why do you think the count doesn't want her to go outside the cabin?
i believe it's because of fmc's mother. as you said yourself, she rejected him but he's not in right mind. after her death, he held onto the most related thing to her. her daughter. it's an obsession with her mother. he wants to monopolise her he hates the duke for that reason too, i think count knows much more than everyone in the story.
Spoiler
Next chap:
They're going to reunite with the Duke (to remind, the Duke is not her Father, he is her half brother)