Ashen Garrison Keys Glitch?

So my crew and I did a couple of ashen garrisons for the first time tonight and the way the keys spawn just seems off. On the first one we didn't get enough garrison keys to open all of the doors. On the second we got 2 extra. And despite spending more than 30 minutes per garrison just looking for ashen keys in the furniture we didn't find a single one to open up the wall safes in the vault room. Anyone know what's going on? Is this just how they're supposed to work or were we glitching somehow?