Boyfriend of 3.5 years has lost my trust.

Sup reddit! This is kinda long, sorry.

My boyfriend and I have been dating for 3.5 years, and our relationship has had our ups and downs but we've seemed to tough out most of it. However, i'm having pretty bad trust issues lately. He has a group of female friends hes been close with since high school, and i've always been okay with them hanging out. 1.5 years ago he admitted that he was having feelings for one of the girls in the group, which was pretty shocking to me. However he promised me that nothing was going on and that they were just friends, but continued to hang out with them (and her). Ever since them i've been pretty uncomfortable with him hanging out with her, even though I know it's not the girls fault at all. I'm pretty much over the whole situation and very trusting of him until...

Fast forward to a year later.. I'm on vacation 6000 miles away with my family and he meets this French exchange student in his class. He tells me shes pretty and nice, i'm whatever about it, it's fine. Couple days later, he springs on me that he wants to break up cuz he can't stop thinking about her and wants to see where it can go with this girl, after knowing her for 3 days. I tell him it's okay if he sees other people, but he definitely won't hear from me ever again. He decides to stay with me, but i'm really traumatized and have lost my trust ever since with him. The thing that happened 1.5 ago has resurfaced, and I feel like i'm back to square one. I can't stand him hanging out with those girls, it's extremely unsettling for me. Seems like whenever a mildly attractive girl shows him any sort of attention, he becomes infatuated and completely blindsided and wants to move on.

My biggest fear is being hurt again, and i'm lost. Sometimes I wonder if I was an idiot for continuing to tough it out.

tl;dr: Boyfriend developed feelings for girls twice, stayed with me, but I am traumatized and afraid of being hurt again. I've lost all trust.