Drifting

I've been doing military style land navigation for a few years and I'm ok at it. However I always tend to drift to my left and end up decently off target. I make sure to put the "red in the shed" and I've even tried setting up an offset to compensate for the drift. Anyone have tips or more techniques to fix drift?