Open matchmaking playlist - selection bias

I hope this is something IW are already aware of, but by creating an open matchmaking playlist, but not rolling it out universally across all public playlists, you are not proving whether that matchmaking system works or not, because of selection bias. Only good players will choose to play it.

Skilled players will select it, because they hate sbmm and sweating every game. Unskilled players and the middle of the range players will not choose it, because they have no reason to. So what you get is a playlist that only appeals to the skilled players, meaning the playlist is super sweaty. The skilled players end up hating it because it feels more sweaty even than quick play, and IW get a bunch of data that says people don’t enjoy that playlist.

Now the key thing is that you cannot use that data to ‘prove’ that the community doesn’t want more open matchmaking. At best it is a fundamentally flawed experiment, and at worst you’ve just rigged it to get the results you wanted the whole time.

This is a poorly designed experiment. You cannot test matchmaking preferences in this way. A matchmaking system has to be applied across the whole of the game, or not at all. I am waiting for an inevitable IW post calling out the data as ‘proving’ their hypothesis that sbmm is preferred. I would just like to point out that would be a rigged conclusion based on bad data. You’d be better off without an open matchmaking playlist. A much better test would have been: weekend 1: normal SBMM and weekend 2: Open. I hope they did that in the background, but I doubt it.