Loan from 401K to Increase Home Downpayment?

I'm tired of renting and my goal for next year is to have enough $ to make a sizable downpayment on a home (most likely a condo) so I can start building equity. I'm 27, employed with an $84k salary, have $20k in a savings account set aside for this downpayment, and have another $40k in my 401k account. The homes I'm looking at are in the $250k - $350k range and my current rent is ~$1500/mo which feels manageable as I'm able to continue to save.

My question is whether or not it's wise to take a loan from a 401k account to assist with such a downpayment. I feel like an extra $10-15k, on top of what I'm projecting to be $25k in savings by the end of the year could make a big difference in 30 years of mortgage payments but I understand how a 401K grows over time and I'm unsure if it's a good idea to pull out such a significant chunk of that $ early on.

Any advice is appreciated!