Holosun 507 Comp moving?…

I have a Holosun 507Comp on my Kimber 2K11 comp, and it works great. Today I noticed the optic plane of my Beretta was loose, so I decided to check all mi pistols with my Boresight.

When it was time for my Kimber, I noticed that the elevation adjustment screw would make the dot go down when I pressed it with the tool, and bounced back when I removed it. That basically makes it impossible to zero. Is that something that happens with the Holosun, or is there something wrong with mine?