First time dating, need some advice!

Hey TwoX! I'm an undergrad in college and recently started talking/dating a guy for the first time. It is apparent that he really likes me, but I don't think I'm on the same page and it's starting to freak me out a little bit. When I talk to my friends about it, they say that since I've never been in a relationship it probably takes me longer to develop feelings for someone (He's was last in a LTR). I've been in turmoil the past few days trying to figure out my feelings. When he asked to hang out yesterday, my immediate reaction was "I don't want to." I'm not sure if this is me freaking out and pushing him away because I've never been in a relationship, or me genuinely not interested in him. Has anyone else experienced this? How did you separate yourself from the situation and figure out your feelings? In the past when I've had crushes on guys I definitely knew. Thanks!