AIO because I asked my boyfriend to not go out with his friends

I(18f) asked my boyfriend (18m) of nearly 2 years to not go out with his friends due to some other girls who were invited. I was also invited but declined because one of the girls is best friends with 2 people who I used to be friends with around 3 or 4 years ago. These people went out of their way to basically make me miserable, to the point of making fun of my mum and brother. It seems silly but when you're 14 and just lost you're only friends over something that seemed small to you, it's major. I barely remember what happened but I had complained about one of them on my twitter account and they found out. Honestly, it doesnt matter much to me anymore but the thought of my boyfriend possibly hanging around these people makes me feel so uncomfortable. I have terrible anxiety over my boyfriend, constantly scared he'll leave me but I know he wont. We have really great communication and he's told me he's completely fine about not going out this time, but I feel dreadful. I feel like I'm not letting him see his friends despite that he sees them all the time lol. I just need some advice and to know if I'm being dramatic.

edit; I just want to add, he sees his friends at-least a couple times a week, I've never once told him that he couldn't go out. All I said to him this time was that I felt uncomfortable with the situation but he could still go out if he wanted to but I wasn't coming and he decided to see them next time.