They are only playing defense and no attack at all. Duchess has been plotting and attacking her and all she has done is defend herself with no harm to duchess reputation. It is also clear that having a mistress doesn't affect duchess in the way they wanted it to, so they need to start plotting and come up with schemes to make her divorce him.
The conflicts, the married life and what's it like having multiple kids are being portrayed realistically in this season, which I absolutely love.