Build a home using half the banks money or pay cash and hire a separate builder to inspect every 2 weeks?

I built two garages before and got screwed by the contractor both times. I want to involve the bank for a $600,000 single-family home build(FL) because I want them to oversee the process to make sure everything gets done properly. ( I don't live in the area)The bank is being overbearing and a pain in the butt. I have half down payment but could pay cash for the build. The bank is gonna get around $15k plus in closing costs, should I give my local builder friend 5 to 8 grand to oversee the hired builder and project? Am I overthinking this? I'm just trying to cover my butt. Truly hate this process. TYIA!