Bf defensive about our weekend together

I wanted to ask you guys about something that recently happened to me. My bf (35y) has his daughters (13 and 15) 50/50 with his baby momma and every other weekend. This weekend was supposed to be ours but his baby momma asked him to stay with the kids bc she is going to a trip with her husband, he said yes. I told him I was excited for the weekend with him so we could do something together and now I was a little sad because we couldn’t. He got very defensive about that and said: “ this is my kids home and they can come here whenever they want to, you have a boyfriend with kids and I am always gonna be a dad first, and you need to understand and respect that”. I felt a little sad about that, I dont think I never said anything disrespectful. What do u guys think? Was I wrong to even mentioned that?