38 with $3.7M: Math Says Retire, Nervous to Do So

Longtime lurker, occasional poster on this sub, but throwaway account as some family knows my reddit handle.

I have ~$3.7M in investments, a $550K mortgage, and no other debt. I'm a 38 year old divorced parent of two young kids-- no child support, we split things 50/50. My investments are largely index funds with ~$2.1M in a general brokerage account, ~$1M in IRAs, ~$220K in 401k, and ~$500K in some inherited IRA's I need to liquidate in the next 10 years. And I have about $50K in cash on hand.

My annual expenses are about $50K if you don't count my mortgage, a little under $100K if you do count mortgage payments (I'm on the fence if I'll just pay it off, my interest rate is about 6%).

In short, I'm working a job I generally like, but this spring I was moved under a supervisor that's really starting to rub me the wrong way. My fire plan has always been "in another 1-3 years", but after the latest dust up, I decided to take a closer look at my numbers. Basically, I think I can fully retire now, but I'm incredibly nervous/stressed to pull the trigger.

My plan is to stay full time through the end of the year (I get about a $15K bonus most years) and also keep an eye on the ACA subsidies to see if Congress extends those. Next year I'm thinking of asking my boss to drop to 2 days a week, though there's no guarantee the company will go for it.

What are your thoughts? I feel like if it was someone else posting I'd be like "GFY, you're good to go", but when it's my own situation, I'm not nearly as confident and moving from the accumulating phase to the spending phase is a huge mental hurdle.