Me (27m) and my ex (23f) butt heads over our child and it's causing me severe stress and depression

I dated B for a few years in my early 20's and we had a son together. We broke up midway through her pregnancy but I remained present in her and my sons life throughout that time. Around the time of his birth B became very verbally and physically abusive to me to the point where we were arguing every single day. I was working a minimum wage job to support everything while also waking up with the baby at 3am because B wouldn't.

Fast forward to today. I see my boy about every week or 2 and that situation has worked for us because of our schedules. But B won't stop being randomly abusive at times. She degrades me, calls me a deadbeat that won't pay for my son (But I will and do often), lies about how she says she will text and call and let me know if I can come over and then doesn't and blames me for not calling or texting to see him. It's gotten to the point where the thought of talking to her makes me severely bed ridden and depressed. I get anxiety whenever I receive a text or a call and it could be her. I find it hard to breath when I go over to see my son.

It's complicated but I signed away my rights when I was younger to facilitate him getting disability instead of doing a custody thing first. I believe I was misled into doing that, but it happened and it's over with.

My main issue is that B becomes insanely irrational when she is upset and will make up things and yell and hang up and spam me, all within a few minutes of each other. Her boyfriend helps calm her down a little bit, and he and I get along, but shes impossible to work with. It's gotten to the point where the thought of seeing my son is depressing me because she will harp and yell and argue the entire time I am there.

I know reasoning with her is off the table, and if I forget a date or don't call and text and ask about my son she will fly off the handle and end up not letting me see him for an extended period of time.

I also won't lie and say the thought of cutting her completely out of my life hasn't crossed my mind. I love my son dearly but my relationship with his mom is causing me to fall apart and miss work and think scary thoughts. I've never had an "antagonist" in my life such as this, since I typically surround myself with reasonable and loving people. I feel like I have an enemy that lords my son over me like a bargaining chip.

I'm somewhat asking for advice from people who were in similar situations. I'm completely lost, heartbroken and feel alone in this. It's gotten so bad that I have thought about moving far away and changing contact information just to be free of this. And it worries me that the constant arguing is having an effect on our son.

Would I be the devil to cut contact with someone who drives me to bad thoughts? Is it wrong to feel this desperate? I want to be in my sons life but I feel like I won't last long having to deal with her. I really need help because I cannot talk about this with anyone in my life.

TLDR: My ex drives me to think terrible thoughts but we have a son together, need advice how to navigate.