Only read until Ch 74 but here's some spoilers : ML lost his memories and his memories back at the time when he has not marry FL (means his trauma bc of his nanny was there too), but his capability to work as a Duke did not affected.
At the same time FL found out that she's pregnant. The elders trying to provoke ML and FL relationship, they trying to drag oriFL into it, but as we know oriFL is a good girl, she won't do anything that could hurt FL. so I don't think affairs would happen.
As for ML, he did not believe the elders easily but also have doubt regarding FL, bc FL influence is way too much as a Duchess. That's the only spoiler I could share.
I was able to read one of the comments here that ml will have amnesia because of an accident. Can you give me some spoilers? What happened after? Is ml and ogfl will have an affair? What about Erin? What will happen to her?