You scare me a little?

So I’ve been dating a man for about 3 months now, and lately he’s pulled back. We met up last night for dinner and we were talking about us and out of nowhere he said, “I’m not gonna lie, you scare me a little.” He rubbed my leg and I smiled and said “how so?” He just started talking about life and other things quickly kind of changed the subject. Is this bad/good? He’s a fantastic man and I don’t want this to end. Do I need to pull back or what? Any advice please. We both work full time and see each other weekly spend time together and communicate great.