Holiday not paid out when left a job (England)

I recently left a job. I carefully ready policies and the email saying I should use holiday unless otherwise agreed. I didn't use up my holiday as I wanted the payout and understood from what was being said that I didn't *have* to. Annoyingly, I don't remember the exact wording of the acceptance of resignation email from HR which made reference to using up holiday.

It was a busy period for the team too so I was legit needed.

They are now saying they don't pay out holiday unless agreed in exceptional circumstances so won't be paying me 2 weeks unused.

Are they allowed to do this? Does it depend on the exact wording of the email or is it clear cut? I have asked them to forward me it and said I was not aware that's what they meant.

Employed for 8 years.