I withdrew £8K from my LISA and am angry about the fees

Even though I knew what I was getting into with the LISA, when I realised I might want to buy in London in the future, I changed my mind in order to not lock away even more money. It just feels punitive for the government to take some of the money I put in (I think about £300). Was this a bad idea or am I correct in wanting to avoid the trouble of overly strict rules later down the line?