Issue with Witch’s hideout

So I made the mistake of going to the Wayward Woods Witch’s hideout and placing the keys at different times not realizing they despawn and don’t go back into your inventory. Does anyone know if there is a way to get those back or is it a bug that I hope will get fixed