Confused about sexuality

I (21f) have been having some troubles with my sexuality lately and I have had a hard time rationalizing it or coming to any solution. There aren't really any people I can talk to about it. I've been lesbian for pretty much as long as I can remember. All my crushes have been girls, I've only really been interested in girls and even when I have that really weird dream where you live your entire life and marry someone (only to wake up and be sad), that spouse is always a woman. So.. definitely lesbian. Except, I've become very close to one of my best friends, who is a guy. I don't know if it's because I haven't been in a relationship for a long time but on occasion I'd just imagine myself doing romantic things, holding hands, pretty innocent stuff. I figured it was harmless and that it was simply because I missed being in a relationship. However things got complicated when he actually confessed that he had grown feelings for me. It came as a shock but we talked about it and I let him down as gentle as I could, but I felt awful. Truly awful. I've had people confess to me before and I could just let them down gently and move on, stay friends, all good. but I don't know if it's because I know him much better than all the other people, but it really hurt me and made me question my entire sexuality. There's a part of me that wanted to say yes, and that's the part that I don't understand at all. I don't find men all that appealing and in terms of sex I don't think I'd ever feel really happy in a relationship with a guy. But there's still that romantic side that's stabbing at my heart and I'm really confused. I genuinely can't make sense of it. On one side I want to say yes to a relationship that is most likely never going to work out and we'd probably both be unhappy in, but on the other side it really makes me sad to say no and really hurts and I don't know why. I feel like my brain and heart have become tangled. In the end I decided to follow my gut and try to let him down gently and move on together as friends, but I just really don't understand why it hurts me this much. Any advice is greatly appreciated. thank you