what to do if one or a few alters want to break up with him but others dont?

title. some people in here really dislike him, but when i go to break up with him i start switching and breaking down and begging him to stay because they love him so much. what do i even do...