Keeping non-combatants out of harms way during combat?

When my colony got to about 12-20 people, I started having the issue that during a raid, I only needed maybe 6-8 defenders at any given moment. I wanted the rest to keep doing their thing. Rest, east, haul, whatever. I have walls and defensive positions, and keeping everyone at attention while I wait for Centipedes to crawl in to attack seems silly.

However, some madmen like to run out the killbox and try to haul dead bodies, items I marked for hauling before combat started, hunt turtles, etc. The only fix I've found is to draft everyone, or restrict the allowable area for each non-combatant during combat. The former make everyone stand around and get angry that they can't eat, the latter is unmanageable.

It put me in mind of Age of Empires. you had a Town Bell that told all your villagers go get into cover.

Is there something like that I'm overlooking, or a mod that can stand in to allow somewhat normal operations inside my base while defenders do their job?

TL;DR: May pawns want to hunt turtles during combat. I'd like a way to keep them safe without drafting the entire colony.

Edit: Here is what I'm envisioning as a possible (moddable?) solution if no vanilla solution existed. Each pawn has two 'allowed' zones. The first is when things are normal, the second is when the "Town Bell" is rung. This could allow pawns to continue to work in safe areas, and gather defenders where you want them before you draft them.