I had a friendship with a guy who became racist

For transparency sake I'm a black Caribbean guy. I lived in Europe ever since I turned 13. 11 years later I basically feel like I gave my energy to a friendship that fucked me up more than my primary school teacher did for my trust in adults

I befriended a guy who was kind and a bit chaotic. To a sense that he always brought me to situations I didn't always asked for

3 years ago after I started to heal physically (I had surgery the year prior + I got a side job alongside college the year after) he started doing racist jokes. At first I didn't think much about it because those kind of topics are not uncommon. My mistake was brushing off personal attacks masked as banter

A friend reminded me of a time near new year's eve where he said he was going to bring my brother over (I don't have one mind you) and he brought a plushie of a monkey. There was also times where he would call by my name other black guys he saw. I also have proof over the years that his girlfriend hated me and when she basically called me a slave I felt bad and was genuinely upset

Our friendship stopped because I confronted him about what he did and he tried to gaslight me. Like seriously making this as something I was blowing out of proportion when it was what I saw. To make matters worse he also tried to weaponize our friendship as well. Everything I explained there happened in 2024

With insight I basically feel bad for not standing up for myself more aggressively than I should. I'm not soft but I always tried to be calm and collected when I express disagreements. I understand now that it's no use being like this because they will always pretend they don't understand that (little trivia : is Asperger a justification for not understanding why I don't like that this is the ONLY joke made and it's always at my expense ? I genuinely want to know)

I don't understand why I can't be mean to people who hurt me. Why do I feel apathy and don't try to get justice when it's possible ? If I have another friend like this he's getting dropped immediately this time