How to tell someone you don‘t want to invite them?

So, I have a friend I’ve known for almost 10 years. We’ve generally gotten along pretty well, but lately he always assumes he can join me for every event I go to. His clinginess has started to really stress me out. Whenever he’s not invited, I actually have to justify myself to him.

Now I want to invite some friends over for New Year’s, but I don’t want to ask him, because he won‘t talk to anyone but me, and kill the vibe of the group. He already asked what I’m up to, and I know that I won’t be doing anything with him, but he doesn’t. So in case he asks again, which he certainly will, how can I politely tell him that this time he cannot come, without causing any drama?