How do the new army controls work with a controller?

I haven't played in a while, and maybe I'm retarded, but this seemed way easier before. If I want to give direction to one group, it seems I have to use the left bumper then select 2 or more groups with the right bumper then use it to deselect units so only the one I want is active?

Like if I had archers selected the last time, I can't just hit LB and select infantry. I have to highlight both, then deselect archers. There's gotta be something I'm doing wrong.