Should I put more down on my new construction or invest the difference?

Bear with me. TLDR: Would you put more down for guaranteed lower payments, or keep the down smaller and invest the difference? How do you personally think about this trade-off?

I’m in my early 40s, reasonably stable job, kids approaching college. I value both lower monthly costs and liquidity, but also want to make the smartest long-term move.

I’m buying a $630K new construction home and debating how much to put down for a mortgage 5.5-6%, 30 years fixed. The reason I’m asking is I see so many people advising/preferring mortgage payoff at these rates.

• $130K down, with a loan ~$500K with ~$3K/monthly payment;
• $200K down, ~$430K loan with ~$2.6K/mo;
• $250K down, ~$380K loan with ~$2.3K/mo.

Per my calcs, I see it’s about $400–$700/month difference, depending on how much extra I put down, with lifetime interest savings of:

• $81K by going from $130K to $200K, or
• $139K by going from $130K to $250K

If I keep the down payment at $130K, I’d have an extra $70K–$120K to invest in the stock market or a business. Investing $70K for 30 years could grow to: ~$227K at 4%, ~$402K at 6%, ~$704K at 8%. Alternatively, investing $120K for 30 years could grow to: ~$389K at 4%, ~$689K at 6%.

So the numbers say investing the difference is much stronger long-term. But the trade-off is:

• Bigger down = guaranteed 6% “return” via interest saved, lower monthly obligation, peace of mind.
• Smaller down + invest = higher payments, more risk, but more liquidity and much higher upside.

Am I wrong thinking min down payment is financially a better solution long term? Anything I’m missing? What’d you do/recommend?