Shuffle Play

Every time I go to the Replay list in my library and select shuffle play, it does not shuffle them at all. It plays the same songs in the same order every time no matter how many times I have selected shuffle.

Any tips or pointers on how to actually make it shuffle?