Trans and gender-neutral?

So, I was AMAB, have some masculine traits mixed with some feminine traits. (Broad shoulders but also wide hips, very tall but an androgenous face, etc etc.) I know I need to be female-bodied, but I don't think I'm female. I don't think I'm really any gender at all. The idea of gender- any gender, but especially the gender binary- being applied to me makes me feel sick and depressed, because it isn't me. I've found myself being incredibly jealous of trans men and other, female-bodied, gender-neutral people. Is this a thing? And is it best to admit this to my doctor, or would this harm my chances of getting more 'mones and eventually SRS?

Ninja edit: I don't mean to imply that trans men aren't "men," I'm jealous that they were AFAB. I know it isn't right, but I'm just talking here, and I can't help how I feel.