Will I regret a $1.12M mortgage?

I should clarify the question-- because we can afford it-- do you think this is DUMB?

The house we are considering is $1.4M.

Income: Our base salaries total ~$450K (soon to be closer to $500K). A lot of comp is tied up in bonuses, which are reliable but fluctuate. In the last 5 years (when we really started making money) we have grossed a total of around $650-$850K (excluding one exceptionally good year).

Expenses: We (40M and 42F) are doing well. We have around $2M saved and no debt other than our current mortgage. This is in large part because our current house is much less expensive. But our expenses are high- we work a lot so we prioritize things that save time (groceries are high, house cleaners, etc.), fitness (gym/trainer expenses) and travel. We also donate a fair amount of money. Truthfully I don't really want to change any of that. We also recently had a daughter, which means daycare, and expect that we will be paying for private school tuition.

Retirement plans: My husband would like to retire early (by 60), but I imagine I will work longer.

I think if we bought the house, the money would mean less savings. We can probably still save around 20% of our gross income without changing much, assuming our salaries continue as expected. If one of us lost a job and could not replace it, we would be hurting, but we would be able to figure it out.

I love this house, and I just cannot see our current house as our forever family home. But my concern is that while we rarely think about money, we will potentially need to budget if we seriously upgrade our house. And, we both grew up with nothing and are generally very risk averse. I am scared to regret it, but I also don't feel the need to hoard tons of money unnecessarily.

Has anyone been in a similar situation? Were you happy you upgraded? Do you hate being tethered to your bigger mortgage? I would love to hear from you!