Being ugly means people want to like you less

Every time someone does something kind, how well it's received is influenced by how much the subject wants to like you. This in turn is directly related to your attractiveness. This is the Halo effect.

Every time you as an ugly subhuman make a normal mistake (be a bit selfish, say the wrong thing, do something slightly inconsiderate), they'll remember it forever and will associate those negatives with you rather than let it go as they do with their more attractive "friends".

Don't believe me? Just try it out. Get an attractive person to do the exact same shit you do, both good and bad, and see who people like and who they remain ambivalent to/dislike

You will always be on your toes around your "friends" so they won't hate you because you know it won't take much.