Scope not holding zero

I'm fairly new to scopes so maybe I'm just unaware of how easy it is to knock them off zero, but I've got a vortex crossfire 2 that seems to come off zero at the slightest bump, like if I set the gun down in any manner other than treating it like a live explosive, it gets knocked off zero. I bought a torque driver specifically to make sure I had all the screws properly torqued and the mounts are only touching the tube itself, nothing else. Am I missing something? Is the scope defective in some way? Or is it really just that easy to knock a scope off zero?