Would I be wrong if I let my child's father just get full custody of them ?

I've been thinking about signing my rights over . He literally wants to be in control of my life and always looks at how I raise my kids and what I do with them . They are currently under my roof but me and him have not been getting along well and I recently just seen he does nothing but kusts over women . I want nothing to do with him . He talks about me and the way I live the way I move with my kids to his first babymom all of a sudden and now I guess in his eyes I'm not the top notch parent . Before we fell off I was woman I was good I was deserving of everything and his first baby mother was bad and not doing nothing with her life etc . I see now that he is just drama and I seriously don't wanna deal with him for the next 18 years of my life . I hate the way my life is going I've been so wrapped around me and him that I've been forgetting about the kids and instead of the father being supportive and stepping back and looking at this all like hey we are a mess he's making it 1000 times harder for me . Telling people I don't have an education telling people that I don't teach my kids they're abcs 123s like I have no clue where this is coming from but it's all lies .and secondly he does nothing ..so it's just like I don't wanna deal with this I have a 7 year old that I've dealt with for more than this and I'd rather go back to just living in a peaceful life with my 7 year old . Did I mention I'm pregnant again ..so either way he can keep the two I have with him and I'll keep my newborn and my 7 year old .