AITA for going back on my word?

Hello all,

My husband asked if his cousin could move in with us. We currently have one of his other cousins living with us as his parents move 19 hours away and needed somewhere to go. We have one child together and one on the way. We have one room that I was planning on turning into our nursery once I found out I was pregnant, but several months ago we were chatting about him moving in too (I was joking but I guess they weren't? I'm neurodivergent so I can't sense when people are being serious or not sometimes.) Well, now he wants to move in New years and we have our baby due in January. He's a really nice guy, respectful, would pay rent, and good with our child when he's around. AITA if I say no now? What should I do? I want my baby to have their own space too.