Should I payoff mortgage?

32 yrs old, we have $600k owed on mortgage at 3%. We are in a position to pay it off. We’d have to liquidate some investment accounts and maybe sell our rental property, both would incur large tax bills. Has anyone here regretted paying off their mortgage this early? I want to do it but am worried about lost wealth from selling the rental property and liquidating investment account that would compound over time.