I am feeling so much guilt and anxiety

I am going out of town this week with my mom for a women's retreat. its been planned for about 6 months and my husband and I talked it over extensively. I'll be gone from Thursday to Sunday. so 2 whole days I wont be here. I'll be almost 3 hrs away. I've never been that far from both kids without my husband. And I feel guilty as heck. He's the stay at home parent so this feels just..mean to leave him with the kids like this. Everyone, including him is telling me it'll be fine but the boys 4, and 20 months, have been extra lately. Screaming and clingy to both of us. I feel guilty for knowing I'll be having some fun and he'll be stuck with the kids by himself for such a long time. I am taking my off days to let him basically do what he wants in terms of 'me time'. And the day I get back he can go hide in the nerd cave for the rest of the day. but I still feel bad. I dont know if I'm looking for advice or sympathy. I just needed to get it off my chest I guess. Half of me is excited to go and the other half is dreading it.