How do you handle it that gay guys are excluded from being guys

im a gay guy and it seems our sexuality changes the way people preceive our sex. you know theres a difference in peoples minds when you say men or gay men.