My BF (29M) won't be intimate with me (36F) if my cat is in the bed

My (36F) boyfriend (29M) told me that he won't be intimate with me if my cat is sleeping in the bed with us.

Some context. My boyfriend and I (let's call him Carl) have been together 1.5 years, and when we started dating he told me he wasn't really a cat person. At that time, I had two cats, and I made it clear that my cats and I are a package deal. I've had them for 10 years, they are my fur babies. Carl accepted that, and although he didn't like them, he learned to live with them. My cats must have sensed how he felt because the dislike seemed to be mutual.. they were skittish and defensive around him, and on occasion if he got too close, one in particular would take a swipe at him.

I used to let my cats sleep in the bed with me every night, but when Carl was over, I tried to compromise and didn't allow them in the bedroom while we slept. I didn’t mind too much because it wasn't every night, and I understood that Carl wasn't used to it. I'd also had phases of not allowing them in the bedroom in the past, because they would meow and get into things at night, every cat owner understands this.

Then Carl moved in about 7 months ago. Now the cats were never allowed in the bedroom. I was fine with it for the most part, plus things started to get better when we decided to have Carl become the primary feeder. He fed them twice a day and it really seemed to help them warm up to him, but not completely trusting. On occasion they would growl and hiss if he startled them.

Sadly, last week one of my fur babies passed away suddenly. This hit me really hard, and Carl said he would never keep my other cat from sleeping in the bed again. I was glad to have my other cat close while I've been grieving and relieved that Carl seemed to feel more comfortable with it. Also I should mention, my other cat has been much more calm and affectionate with Carl since Mittens passed away. I'm not sure why.

But last night, I tried to initiate (you know) with Carl while my cat was laying in the bed with us. Carl said that he didn't think he could be intimate with a cat in the bed, they are dirty and he wouldn't be comfortable. He said that he's been having trouble sleeping too. I backed down and said obviously I don't want him to do anything if he's not comfortable.

But I'm just confused and hurt at this point. How do I choose between letting my only fur baby sleep with me, or kicking him out so my boyfriend can be comfortable? I care about Carl's comfort but I'm just not ready to close the door on my other cat again.

This may be a silly problem but I'm still feeling so sad about loosing Mittens, and now I feel I have to choose between my comfort and Carl's.

I'm just looking for an objective opinion at this point, because I feel like I'm not seeing the situation clearly, and I don't know if I'm being unreasonable.

Thanks in advance for any advice!