How do you switch active thrall crafters for a bench?!?

This is driving me nuts. New to Conan Exiles and working on my first big base with captured thralls.

I have an Armorer and a Tanner and another thrall I captured. Some are Level 2, some are Level 3. All have different bonuses as far as I can tell.

How do I make my Tanner work at something like the Improved Tanning Bench instead of my Armorer? I know it has nothing to do with who is graphically showing at the bench, it's about the little box in the crafting area.

I can't click on that box or right click on it or change it in any way. It's always the Armorer, who was also coincidentally the first crafting thrall I grabbed.

That Armorer seems to take over every vaguely related bench and I can't figure out how to swap them out. Makes it feel pointless to get more crafting thralls when the one Armorer just keeps taking over everything and there's no way to assign different ones to different stations.

Any ideas?