Shuffle resets if I skip a track

Has anyone noticed that? If you play music on shuffle and after listening to a few tracks you skip one song, one of the songs played previously plays again. Feels like the whole queue just resets and reshuffles the whole folder.