How do you handle it when one spouse wants to go to bed together and the other doesn’t?

For couples who have different sleep schedules or different preferences around bedtime, where do you draw the line between reasonably accommodating your spouse’s desire for closeness and still being able to decide when you personally want to go to bed?

Context:

We’ve been together 12 years.

Sometimes I (38f) don’t want to go to bed when my husband (41m) does. Whether it’s because I’m reading, watching something, or just enjoying having some quiet time alone, I’m not ready to go to bet yet.

The problem is that my husband seems to get genuinely agitated when it reaches the time he considers “bedtime” and I don’t follow him upstairs.

In the past, he’s done things like turn off every light downstairs while I’m still sitting there, leaving me in the dark. Other times, he’s turned off the TV while I’m actively watching it and announced that it’s bedtime. I’ve talked to him about it and explained that I don’t like feeling as though he gets to decide when I go to bed. He acknowledged it and said he would work on it.

There’s another part of the pattern, though. If he goes to bed first and I come upstairs later, he wakes up and becomes very physically affectionate. He’ll insist that I tell him I love him, want me to lie on his chest, or practically lie on top of me—one arm underneath my neck, another across my chest, and a leg over mine. It can become a whole interaction rather than me quietly getting into bed. It’s smothering. Then the next day, he’ll tell me how exhausted he is and say he needs to start going to bed earlier because he isn’t getting enough sleep. He’s also tends to make “jokes” about how it’s my fault.

He’s a very light sleeper, so I’ve offered what seems like the obvious practical solution: if I want to stay up later, I can sleep in the guest room so I don’t wake him when I eventually come to bed. He is very insistent that he does not want me to do that either. When I’ve done it anyway, he makes his displeasure obvious the next day by doing things like giving me the silent treatment, acting distant, or telling me about how I disappointed him by not coming to bed like I’m supposed to.

I get wanting bedtime to be a shared ritual or wanting physical closeness with your spouse. I don’t think there’s anything inherently wrong with that. But I’m starting to feel like there isn’t actually an option here that allows me to choose my own bedtime without it becoming an issue: if I stay downstairs, he’s unhappy. If I come to bed later, he wakes up and engages with me and then complains about being tired. And if I offer to sleep separately so I don’t disturb him, he doesn’t want that either.

tl;dr: My husband wants us to go to bed together, but I don’t always want to. If I stay up, he gets upset; if I come to bed later, he wakes up and smothers me with affection, then complains he’s exhausted. He also doesn’t want me using the guest room. How do other couples handle this?