There's something fundamentally wrong with skill ratings, and I've got some math to show it.
The devs might know about this already, and I know there was a post a few days ago by one of them about how they're looking in to different ways to do skill ratings, but I just really want to make sure they're aware of this.
So the basic problem is that it takes way too many games to increase or decrease your rating. The guy I play doubles with and I always either gain or lose around 1-5 points on either a win or a loss. I thought for a while that maybe we were just being unlucky and only beating people ranked way below us and only losing to people ranked high above us, but I started to think that's not the case and looked in to it.
Here's what I found: Last night we played a match where our ratings were 1123 and 1043 (Avg 1083) and our opponents were rated 1017 and 1328 (Avg 1172). We won and each gained 4 points.
Assuming the other team also lost 4 points, that means we would have to beat that team ~10 times just to have a similar rating to them (assuming we always got 4, which isn't the case as we'd probably get fewer and fewer as we got closer to them). And since we can infer that matchmaking treats a difference of ~100 as being the same since we found that match almost instantly, than we'd have to win ~20 just to swap places with them and maybe start to move out of their range.
If you beat a team ranked above you multiple times, you shouldn't be near their rank anymore. You've proven you're better and should be matched against higher players (or conversely they should be matched against lower players), but with the current system that doesn't seem to happen.
Now despite all of this, we tend to have around a win percentage of 50% every night, which is what you want out of a matchmaking system. However that's partly because we often do play teams that are rated way, way higher than us. We've played guys in the top 20 multiple times, so obviously we're gonna lose those. It seems like against similarly rated teams we usually fare pretty well, but at the current rate it has literally taken us weeks to start to get out of the 1000-1100 range.
Anyone else got any insight into this?
Posted Aug 1, 2015, 3:52 PM UTC
I've just started trying my hand at 1v1s I get <10 for a win and slowly make my to 1000 then as soon as I hit that I get barely anything for a win or my rating plummeting for a loss. Just doesn't seem right to me.
Games are pretty well matched though to be fair.