Builder trying to add VAT after fixed price agreed — am I right to push back? (England)

We’re currently in the two thirds of the way through building work.

We agreed and signed a fixed-price contract with staged payments. All stages were invoiced with VAT as normal. Apart from one stage (£19k) was clearly shown in the payment schedule as a direct payment to the fitter (not VAT registered) with £0 VAT, and it was included in the agreed total fixed price. Total cost of build is around 100k

Since then, the business has said mid build that the fitter has apparently become VAT registered. We’ve now been told VAT should be added to the £19k and that we’re liable for it, really putting the pressure on— but at the same time he’s also suggested that to “get round it” we could just pay the £19k in cash….

Very reluctant to pay cash, as we purposefully chose this company as they seemed legit, sadly they don’t seem all that strait laced. I’m fairly confident that VAT can’b be added onto that one payment after the fact when the contract clearly showed £0 VAT and the price was fixed and signed. Scanning through the contract it also doesn’t say anything about vat status, other than the price is fixed and final.

Am I right in thinking this is their issue, not ours? Would love peoples views, find it all v stressful!