Is my boss interested in me, or am I reading too much into a slow-burn workplace dynamic?

I work with a guy who is also technically my boss. For months I’ve felt like he’s different with me than with other coworkers, but not in the obvious “constantly talking to me” way.

He’s actually very social and chatty with other people, but with me he can sometimes seem quieter, more aware, or more careful. There have been a lot of moments that feel personal:
He notices things about me (outfits, what I’m eating, where I’m going, etc.).
He once commented, “Where are you heading dressed like that?”
I referenced something about his hometown and he seemed genuinely surprised I remembered. His tone got noticeably softer afterward.
There have been playful interactions and teasing between us.
He has seemed protective of me on at least one occasion.
I often catch him being aware of me even when we’re not actively talking.

The confusing part is that he doesn’t consistently initiate conversations with me, and after some warmer periods he’ll sometimes seem quieter again.

A while back I invited him for a drink. He seemed open to the idea at first, but when it came time to actually figure out logistics, he never followed through and left the message unanswered.

Recently I invited him to a group workout class. He said he doesn’t really like those types of classes, but instead of saying no, he asked questions about it. First he said, “If I’m not working,” and later came back and said, “Yeah, I can probably go.”

I jokingly told him I’d sign him up myself, and he laughed and said, “Just because I sign up doesn’t mean I have to show.”

The vibe felt playful and receptive, which honestly surprised me because I was expecting him to decline.

So now I’m confused.

On one hand:
He seems different with me than with others.
There have been a lot of moments that feel personal.
He seemed more open to the class than I expected, especially since it’s not something he enjoys.

On the other hand:
He hasn’t followed through on one-on-one plans.
He doesn’t consistently take initiative.
The boss/employee dynamic obviously complicates things.

Does this sound like someone who may be interested but cautious and slow-moving? Or does it sound more like workplace familiarity and friendliness that I’m interpreting as something more?