How to remove people from the "people you've played with" list?

So on 360 there's a tab in the friends list menu which is a list of everyone you've ever played with in online matches, but there's someone I had a REALLY bad experience with that I want to remove, but there's no remove button, and even though I blocked them, they're still there

Is there any way to remove someone from the list?