Boyfriend that my family and friends love

Hey yall i’ve been dating this guy for the past almost 3 years. My family and friends absolutely adore him and every one thinks we’re going to get married. My real issue is that his personality absolutely drives me insane. I love him and we’re best friends. But when it comes to having him as a partner he freaks out constantly from being touched because he has sensitive skin. Everything is a joke to him and it’s almost impossible to get him in a serious conversation without him shutting down and going mute. I obviously have my own issues but i’m scared to continue the relationship because deep down i can’t stand him. He does all these amazing things and loves me so deeply but he drives me up a wall. I have kind of a difficult personality so do yall think this is something i can overcome or should i end it?