Garrison bug?

My garrison hotkey works fine at the start of the game, I have it bound to 'g', and i can garrison my units and pull em out easily with 'g' but, sometimes as the game progresses, it no longer works and I need to manually un-garrison my units and that makes my micro hella slow cuz my game style is archers + palaces.

Have you guys got any clue as to why this is happening?