People wanting alone time with my five week old baby

My father in law has “joked” about taking my infant for the weekend since the day he was born. Now my sister in law is asking for us to go on a date so she can have alone time with him. The weekend thing is going to be a no forever or at least til he’s a kid. The date thing will come in handy when he’s older: I’m sure we will welcome it when he is past his early days. But honestly I do not understand the expectation of alone time with an infant. I have made my house an open door for them to come see him. That’s the most I’m willing to do, and to me it feels extremely generous.

I guess I don’t get it because there’s no world in which I’d expect any friend or relative to leave their brand new infant with me so I could be alone with them?? It just makes no sense to me at all. I know this is a thing now and I don’t get it. My family never says stuff like this.