Why am I considered racist for not being physically attracted to men of another ethnicity?

I realize this may be a very unpopular sentiment so bear with me:

So I had a conversation amongst friends about what "type" of guy we are attracted to. We pulled up pictures of celebrities to compare, and some time in the conversation I mentioned that I am not typically attracted to African American men. I find white men and Indian men attractive in most cases. They berated me for about a half an hour on how racist and discriminatory I was. I find this very odd because isn't the only difference we should acknowledge between people of difference races is their physical differences not stereotypes about ability or intelligence, etc.? There is no denying that people of different races have different outward appearances and that sexual attraction is hard wired in our brains for particular characteristics: height, weight, muscle tone. Why does this not extend to skin tone, eye color, hair color? I'm curious how other people view this because I see it as thinking a gay man is misogynist because he is not attracted to the external appearance of a woman.

Note: I am white Other note: This does not extend to personality or internal characteristics. I could very well find myself falling in love or being in a relationship with an African American man. We are talking pure superficial physical attraction.