Only seeing a guy for a week, but he already said he doesn't feel anything and might not be able to fall in love at all...

We met 3 times, he initiated all of them. Each one was hours long and we had a lot of conversations. It was really nice. I just got out of a toxic relationship about two months ago and I had a bit of a panic attack after the first time we met, but I had a good time so I continued.

I'm attracted to him, I enjoy our time together, and I know feelings can take some time to rise. But he already tried to give me the "what are we" talk, in which he realized he doesn't have anything specific he wants in mind, and shared that since having his heart broken years ago he couldn't feel being in love with someone. He broke a lot of connections since, and started taking SSRIs that he thinks helped with the emotional numbness.

He asked me if I'm into him. I said that it will take me some time to open up, but yeah I think I'm starting to be. When I asked him if he's into me, he said no. It felt like he's trying to push me away. He said after a bit of conversation where I explained to him "being into someone" doesn't necessarily mean having a crush on them, but it can be being curious about the connection and enjoy their company. He then said that yes, he is attracted to me, and enjoys our time together, but he doesn't feel like he catches feelings.

My question is, how big of a red flag is this? In the initial "no" I was like, alright I'm done. But when he explained it, it just sounded like he expects himself to fall deeply immediately after a few days of knowing someone. I did have a huge crush on my ex the moment I met him, but I know for a fact those kind of lustful connections can turn out really toxic, so I'm not looking for that myself anymore. I enjoy the slow burn. But if he says from the get go that he can't catch feelings... I don't know. It feels off.