Is my (24F) boyfriend (29M) over his exes?

My boyfriend has had a few relationships in the past (many years ago), and from what I understood, some were ended by him and others weren’t.

I asked him if he’s over his exes to which he replied to at he’s nostalgic and misses them sometimes, so he never truly gets over them in that sense. However, he says he does not and will not contact them.

That hurt me a little because in my mind if we’re together and he’s happy, there wouldn’t really be a need to “miss” them. I’ve never had a relationship prior to him so I can’t say I fully understand, but am trying to. I can’t tell if this is an internal conflict and just something I have to come to terms with, because I know there must’ve been good times in his past relationships too, or if it’s something I should talk to him about or is something I should be concerned about. I think I’m just a little insecure that his heart will always be with some other people.