On the precipice. Better to indulge or resist?

Hello, I'm a mid-thirties married Man with young children. Recently I met another Mom of young children on the playground and we chatted as our toddlers played. We've been meeting up for walks without the knowledge of our partners. And the conversation has heated up, it's explicit that we are interested in hooking up, but haven't yet, only talked about it.

My situation is that I haven't had much sex with my wife, 10 times a year or less for the last 5 years since she has been constantly pregnant or breastfeeding during that time. I'm sensitive to that fact, and my previous position was that I should wait until she stops having her hormones impacted, and then bring up my desires for a more active sex life. The only problem with that is that I realize it's not a more active sex life that I want but a more active "fantasy life". We haven't discussed fantasies in maybe 10 years, because I know she is insecure, and squashed that conversation years ago. I'm unsatisfied, because I think I just am that bit more freaky than her. It may be my only complaint in an otherwise good marriage.

On the other hand playground Mom, is quite lit-up by fantasy talk. But that's easier to do when you don't have a history with someone, I know. I also know that I might be crushing on a new person, but I'm attracted to the adventure of it as much as I am her in specific. I know that if we were to be found out, the cost would be astronomical to unimaginable. I don't want to risk my family or hers at all. There's still some chance that over the next year(s), I could slowly bring up my desires to my wife, and even involve her. Playground Mom is keen to do swinging with all four adults here, and I wouldn't mind any kind of arrangement. But there's also the possibility, which seems more likely, that my wish isn't granted.

I do believe that there's some truth that you owe it to yourself to take what you want from life. There's sacrificing for your family; and there's also giving to yourself so that you can not be depressed for 10 years before rage-divorcing.

My life is acutely stressful right now for other reasons, children, work, elder-parent duties. I'm learning about myself that my go-to guilty-pleasure when I'm stressed is "fantasy" imagination. However the torture of the will-we won't-we affair only adds to that stress. From that perspective, I should try and be less stressed, and part of that should be sealing off the option of this affair. But it's not satisfying to do that - then I'm just grinding my life out. This conundrum is wrecking me, most nights I have trouble sleeping playing out the affair scenarios in my head.

I don't mind playing the long game, I'm a patient person. I'm also quite good at being sneaky, I think I could have what I want and continue to live with it not-found-out.

How should I best play this hand of cards. All-in, or fold?