Hiding How Long I've Had Trans Feelings?

Quick intro. I am a 27 yo ftm, dating another 31 yo ftm. We have been going out for about 4.5 years and I came out as trans to him last year, at first he had a really hard time with it but has gotten over it and has been wonderful.

I told him that questioning my gender was only a recent thing, but I have been questioning it in some form since at least junior high. When we met I was, at the time, doing a pretty good job of repressing and not thinking about it, I was so busy with school that it was sort of "taking the edge off" if you will, so I was able to distract myself from it.

I will also admit that I was worried if I told him that he would think we started dating just because of his own transition and that he might feel I was glomming on to him. This is not the case at all, his being trans is completely coincidental to my own issues with being trans. We started dating only because he is wicked awesome. (And hot as fuck to boot!)

My question is, should I have told him sooner? I feel guilty having not done it sooner, like I was hiding something. Things are going well now, should I tell him that I had gender issues well before we met?