Sometimes I wish my job was the less flexible one

My husband works in office 5 days a week with a 45min-1hr commute each way. He's out of the house about 7:30am-6:15pm. He's in upper management as the head of his entire building so his time off is very rigid and has to be approved weeks in advance. I, on the other hand, work fully remote from home in a very flexible, independent contributor role where I have few meetings, unlimited PTO that my company is really great about, and I can generally move things around in my day pretty easily. I understand it's a desire-able set up overall as a parent of a 2.5 year old and a 5.5 month old. Our kids get up between 6:30 and 7, I handle breakfast while he's getting ready for work. Then I hang out with the kids until our nanny gets there at 8:30. I work from 8:30 to 4:30. Then I take back over the kids from 4:30 until whenever he gets home.

But, I am so tired of being the flexible one. I am so tired of being the one that has to rearrange her daily tasks when our nanny needs the morning off or when my kids need to go to the doctor in the middle of the day. I'll mention Nanny needs a morning off and he'll go "Oh, okay" because it ultimately doesn't effect him. Sometimes I would love it if I got to just get in my car in the morning and not think about what's going on at home until 6:15pm. Don't get me wrong when my husband gets home we are both 100% with the kids until they go to bed. But then after they go to bed he wants to hang out and I just want to exist in silence by myself. He gets 2 hours in a car by himself which he says it's relaxing alone time but it sure would be for me. But then he gets upset and says he doesn't feel like we're a real couple but rather just 2 people parenting together.

Idk, I think I'm just tired overall. I love my job, I love my kids, and I love my husband...but sometimes I need to prioritize myself when I can and all that seems to do is hurt him.