I'm stumped on the Expedition map

I'm on the second map. I've cleared everything except four gold chests and the emerchest. That means that I need 12 total keys still, but I only have one and there are no more to be found, the map is empty. WTF?