Boyfriend (33M) and I (32F) have different approaches to friends. How do we manage this in a way that supports both of our needs?

My boyfriend and I will have been together for one year next month and we have what I feel is the best relationship I have ever been in. We are committed to each other and on the same page of moving in and engagement before the end of this year. We love each other deeply and have worked really hard to get to a stable and happy relationship. I do struggle with anxiety.

Now to our current issue:

My boyfriend has a lot of friends (like 10+) from different groups and from various life stages (college, grad school, etc). I have a few close ones (2-3) and see them maybe once a year but do talk or text frequently. I have never really valued having a large group of friends and am introverted. I prefer the few close friendships I do have. I should note that he has female friends and I am not necessarily opposed to this. We have discussed boundaries and he has so far upheld them. This issue is more about my lack of ability to relate to certain people than gender.

From early on he has tried to bring me into various friend groups. Over the summer there was a beach trip where I met a group of his friends, and I think it went well. He accommodated me and we have since hung out with different members of that group again.

I also get along well with his family and see some of them almost daily while he sees members of my family maybe once or twice a month.

Where I am having difficulty is with his one friend group from college that I just cannot seem to crack. I can't put my finger on what the issue is, but I just feel extremely uncomfortable around them. One thing that sticks out is that one of the girls is ALWAYS late to everything. We also have to travel to see them (1 hour each way) every time there is a get together. Over the course of our relationship, I have met up with this group at least 5 times and he has met up with them several times without me. There would have been an additional hang out, but they flaked on us after we had already driven to their city. This still makes me feel mad.

Last night we drove to them again to hang out at one of their apartments. I felt super uncomfortable the entire time. When we were walking out afterwards my boyfriend was shocked to hear that I did not have a good time. He thought it had gone great and didn't understand why I felt so uncomfortable. To be honest, I am not entirely sure myself. I think it is a combination of being new and an outsider, feeling insecure with some class dynamics, feeling like he acts different around them than around me, the lateness which I consider rude, feeling like people talk over me, and downright not having anything in common with them.

I am not at all trying to get him to stop being friends with them. Nevertheless, when he does go hang out with them without me, I do feel sad and like he is choosing them or I am being excluded. A relevant thing here is that they seem to always make plans last minute so if him and I had something else going on, even just eating dinner/hanging out, I feel like our plan gets thrown out so he can hang out with them. I also feel like some of the friends being girls makes this feel weirder to me because it's not like it's a "guy's night" where it would make sense to exclude girlfriends. I also recognize that it is still important to maintain the friendships independent of me I just struggle with it in the moment.

I am trying hard to find a solution where we can both be happy. This morning after talking more about last night I arrived at the conclusion that I do not wish to hang out with them anymore. The problem is that now every time he goes to hang out with them, I will still feel left out and not chosen.

My question is how do we manage this situation? I know I need to manage my own emotions but I also feel like there is room for compromise and creativity. Would love to hear others' thoughts!