AIO - Husband agreed we would do a family day today but didnt want to do anything besides lunch

My husband usually works Mon through Sat sometimes Sunday. This just started recently with his job and I told him I would be supportive. He have to work those extra hours as im the breadwinner and our bills are covered. This means though im now solo with our kids during the weekends and after we get home from work its usually just me as he is too tired from work. Again all understandable as I understand his goal with working extra time. I just asked that he as a compromise just does family time every few months with the boys and I. I take care of everything else - their soccer clubs, bedtime routines, food prep, etc.

Today was going to be our first family day in a long time. He took off for his moms bday on valentines day and we were there all day. Today I asked him to go to brunch and for him to take me and the boys out. He agreed and I asked him again this morning if he was still okay with that and if we could go to the mall for me to exchange an item and to take our boys to the indoor playground. Then he said no and that he just wants to be home and was complaining. That he just wanted to do what we had to do with brunch and go home.

I naturally got upset as it now feels like just chore to him. That he never ever wants to do with anything with me and his 18month boys besides just be at home. Anytime he works a lot he gets like this. I just said hey lets not worry about chores lets just have fun with the boys just for today. Told him I was off tomorrow I cleaned and can take care of his laundry. Then he got upset basically saying I was tricking him into doing other stuff and that we can never do what was asked. I mentioned the indoor playground because our kids have never been and we were going to be at the mall anyways. i didnt see the harm in spending 10-20 minutes for our kids to have fun and not be stuck at home. He then starts yelling and turns the around the car and takes us home. Saying we just arent going to do anything today and that its my fault for overreacting and asking to go to the park. I understand him being tired which is why I thought ahead with cleaning. Anytime a fight happens he always cancels plans and its starting to feel its being done on purpose to get out of doing stuff. After coming back home he also just sped off and left in his car.

Not sure if I am just making this a bigger deal than it needs to be and maybe I should have just not pushed it and kept quiet when he said no the first time. rather than trying to explain why it wouldnt be a big deal to go for a few minutes