AITAH for not staying for over a week with brother and SIL after they just had their newborn?

So for some background:

My (21F) Brother (27M) and SIL’s (27M) families are both from the same small rural town, however a few years ago they moved a city about 4-5 hours away for their careers and got married. Im the youngest and our dad works away from home so its mostly my mom and I here all the time (currently on summer break from college), we have 3 dogs who are rescues or pandemic dogs and are quite anxious when we are gone.

Two weeks ago they had their baby and my brother who is a doctor had started medical residency only a few days before that, my SIL is currently in maternity leave or isn’t planning on going back to work but we aren’t sure. Understandably with all this new stuff going on they are busy, before the baby was born we had planned to be flexible (which thankfully our jobs allow) and go visit them and help out in any way we could since it’s common for people here to travel to their city.

When the baby was born we were all invited to be there and visit the baby when SIL was in her recovery room, its was pretty exciting since it’s the first grandchild in each side. We stayed near the hospital and went back, now after two weeks SIL’s mom is leaving and we were preparing to go there tomorrow (Thursday), there’s no dog sitting or boarding services here so a friend of my mom is kindly helping us take care go them here since we can’t take them either because my brother lives in an apartment and they already have 2 big dogs.

So today we spoke on the phone with brother and told him we plan on staying until Sunday maybe Monday, and he immediately told us that was “way too little time to be there” and that it’s basically nothing. He said in a pretty casually annoyed tone. He said we should stay at the very least until Monday and that it would still be too little.

We were surprised by this, my mom and i personally thought that they might not even want us to go and want some alone time (which she said she wanted when having her babies and fits most stories I’ve read on here).

My brother doesn’t have a great history with being comprehensive, he has a temper and says things before he really thinks them, we mentioned the dogs but he basically ignored it. We also haven’t really heard from my SIL on this as he is basically the one telling us all this and she is the one primarily taking care of the baby since he did not take paternity leave.

I guess it’s our mistake to assume that they wouldn’t want this but we unfortunately can’t leave our home and dogs alone for a long time, SIL’s mom had her husband and other kids to stay behind. But we are trying to be the most comprehensive and it seems to not be working very well