Should I give a older man a chance?

Recently, I hit it off with a man fourteen years older than me. He showed interest in me and we have a few things in common, but I'm unsure about reciprocating his feelings for me because I'm not actually attracted to him yet. What keeps from me being attracted to him is our age gap, aside from that, he is average looking and I can see myself looking past his physical appearance. Should I try to have a relationship with him or should I keep looking for a younger men? I don't think I have a change with a men of my age because I'm a 4 out of 10 and I have made fun of because of my looks and I have been invisible due to my appearance for quite some time.