I started a federation together with an Awakened Ascendancy, the Watchful Regulators, once the Unbidden came to my galaxy.
From that moment on I could never leave the federation again. Doesn't matter if I'm president or not, I can't leave, disband, do anything to this federation at all.
So I though to myself "alright, maybe they won't let me leave while the unbidden are still a threat. They're always talking about that which threatens us all when I contact them".
So I set out with my measly 60k fleet, thanks to my measly 410 naval capacity that I can't expand anymore because my federation partner will vote NO on every imaginable war I can come up with.
Finally managed to close the port