I'm struggling to know whether I should move on, or go back to him and try to figure things out. If anything he needs to talk to someone (not me) about his emotional baggage that will affect any relationship. 
I just feel like we click so well together, it has only been 5 months and the only issue we had is schedule clashes to hang out.
ugh idk, I've never felt so stuck

I saw so many red flags reading your post. It seems like he is carrying emotional baggage from previous relationships and is projecting on you hard. I would advice against you geting back into a relationship, seeing as you are worth someone who is confident enough in himself and in you. A relationship where you do not have to prove someone elses worth and run after them because they cannot deal with past trauma and see that what someone else did to him has nothing to do with what someone else will do.  
His insecurities ended this relationship and I am sorry that you are hurting because of this. But it is not your job to fix someone who needs to reflect upon his choices and the unresolved emotional pain. He needs to look within himself and heal  
 (Since he keeps compraing you to his exes shows me that he jumped into a relationship way to quick and has not gotten over them)  


Whatever you are gonna do is up to you, this year will end soon.  
Take this time to figure out if you want t

My SO and I had an argument about something minor, and he feels that I'll hurt him like his previous girlfriends (he even compared me and a previous gf, when we have nothing in common).

He always would joke that I was cheating on him when I said things like, "I appreciate you", or if I don't react like a crazy person when he hangs out with his mates.

He called the relationship off because he can't see the point, and "it's not worth it". In my heart, I feel like it is worth fighting for but I'm scared to even try because he has already asked me "what I see in him that proves he can work through it". I just feel dishearted because I care so much about him, and want him to be able to see the positives in our relationship rather than look for any flaw that could hurt him later.

I (f19) feel like my bf (m21) gave up so quickly on our relationship because of an emotional roadblock he created.