Having trouble getting troops in awakening.

How do you guys get your towns to get lots of troops? I always find myself vastly outnumbered by clams with less towns than me because each one has 3-4 times more troops than mine.

If I set my castle town build to "attack" they generally get enough troops but I only have retainers with the right stats for 2-3 towns.

I could build the town building manually but I don't have enough labor to do that quick enough before being gobbled up by my neighbors.