How to not burn a bridge as a consultant starting a solo project but got offer elsewhere?

So as title says, I’m a consultant at a smaller company and am onboarding onto a new project. It will be a solo project where I am the only POC, working with the client on some government regulation. My direct leadership kinda had to “sell” my experience to this client and was told this is a key client, project and can bring in future projects. I’ve gotten an offer elsewhere which I’d like to accept as I’m over my current company (issues with management, resourcing, pay and promotions) but feel like I can burn a bridge and negatively impact both current company and client. Has anyone been in a similar situation or experienced something similar? I know loyalty doesn’t exist in corporate but I don’t wanna put some people I like in a bad place.