Low self-esteem, but self-absorbed?

I was talking to my wife last night and was able to describe this social anxiety I have. I'm more likely to fear what people think and say about me than my own safety. My wife and I talk about this often, specifically about how many women don't have this luxury. I was able to elaborate on my social anxiety and realized that if I come across 1 person who thinks I'm an idiot, asshole, or some other negative trait, i automatically assume EVERYONE feels the same way about me. Then I hyper-focus on trying to fix myself, but i truly never know if I succeed. I just keep trying to fix myself so that everyone will like me. But it's not fair to expect everyone to like me. On an emotional level, that's survival for me. I think I also have abandonment issues but that's a post for another day. So even when I'm around people, i still can be present or effectively read the room because I'm trying to not be the "bad" me.

This is more of a rant than anything, but I'm curious to read anyone else's stories as well.