Jun 20, 2016Hopefully this post will answer a lot of frequently-asked questions and quell confusion about the nature of matchmaking.
What is the intent of the matchmaker?
The matchmaker seeks to fairly match players according to the following criteria:
- Similar skill (MMRs are close)
- Similar ping
- Similar group size
(Note that level is explicitly not included here.)
What is "skill"?
"Skill" is not measured absolutely, it is inferred by predictions and game results. The quality of opponents you meet and beat ultimately paints a picture of "stronger than A, weaker than B". Because individual player skill varies wildly (someone can play distracted and underperform or be exceptionally focused and overperform), it is often represented as a Gaussian curve. The mean of the curve, and what we determine to be the measurement of skill is what is called a player's matchmaking rating, or MMR. It starts at some seed value and, through wins and losses, goes up and down. In-game metrics are not factored because they can be gamed or otherwise manipulated.
Can you give more detail about what exactly MMR is?
The absolute best resource for learning about rating systems are sites that detail the Elo and Glicko rating systems. Blizzard's been using a similar system for years and years. The core concept is that they're all designed around win probability on a per-game basis.
Let's say, for example, that everybody starts at 1500 MMR. When you play against other 1500-MMR players, the win probability for that 1500-1500 matchup is 50-50. If you won, your rating would increase by an amount (a tunable constant called "K-factor" multiplied by the win probability), and you would go up to, say, 1516 while the other team's rating would drop.
The difference in rating for two players translates into an expected win probability. For example, if the pairing were 1600 vs. 1500, that might mean a win probability of 64-36 (the probability depends on how the system is tuned). If the lower-rated player won and produced an upset, that player would earn more rating. If the lower-rated player lost, that player would lose less rating.
I'm making a reasonable assumption here that, since it's a team game, the changes in MMR are applied individually. That is, your MMR compared to the other team's average MMR. An opponent's MMR compared to your team's average MMR. A teammate's MMR compared to the other team's average MMR, and so forth. This would mean that two players on the same team with different ratings would have their ratings affected differently after a game.
How much does MMR change per game? Does it fluctuate wildly?
The degree of change depends on the gap between you and the average rating of the other team. Let's say Overwatch's system uses a K-factor of 32 (very common, and this is what SC2's system used). If your win probability was extremely low--0.01%--you would gain 32 rating (if it was 50% you would gain 16). However, even with the insanely unlikely +32 rating scenario, that still doesn't have a very large impact on your rating as a whole since going from 1500 to 1532 is relatively small. SC2 preferred a range of between 45% and 55% matchups wherever possible, and Overwatch has a ton of active players right now, so that's probably not too far off the mark.
Some systems have a "provisional" K-factor reserved for new players so that their rating can more quickly approach their true skill (SC2 had this as well, and it lasted for a player's career-first 25 games). Generally this is only moderately higher than the standard K-factor, something like 40.
What's the search range for other players?
The search range (in terms of MMR) depends on the system's confidence in your current rating. If it's highly confident due to a high number of recorded games, then the search range would be pretty small, maybe +/-50 rating. If it's not very confident due to a low number of games played or a lot of upsets, then the default search range would be wider.
A lot of systems also tend to temporarily and gradually expand the search range after X seconds if a match has not yet been found. I expect Overwatch is no different.
My teammates pick terrible heroes or fail to coordinate properly. Is the matchmaker trying to keep me at 50%?
The matchmaker determines expected win probability at the time the game is created. It doesn't care that you're on a winning or losing streak. It only looks at your current rating and matches you with 11 other similarly-rated players. Once the game reaches the hero select screen, the outcome is decided solely by the players. It doesn't know which heroes both teams will pick or how well each team will work together.
What happens if I'm in a game where everyone's a similar MMR, but one side keeps losing?
Statistically, it appears to be a fair match, but it's proving not to be as the results consistently favor one side. However, the Overwatch system has a failsafe in place in this case where the match will be disbanded (both sides will requeue and "Finding a more balanced match" will appear) once a threshold of consecutive losses is met.
Note that this does not apply on a per-game basis, but rather across a series of matches with the same participants.
I'm a solo player and I'm always matched against premades. Shouldn't the matchmaker force stacks to play other stacks?
It's very easy to assume that you are a group of individuals playing against a 6-stack. You know that you queued alone, and you see that the other team coordinates exceptionally well. One thing to consider is that there is a high probability that you have a stack on your team, too. That "solo vs 6" is very probably a "1+5 vs 1+5" or "1+1+1+3 vs 1+1+1+3".
Currently it's difficult to prove this, however Tigole mentioned that in the current internal build, Competitive Play will show which players are grouped together.
Why do I see level 100s and level 1s on the same team?
There are many possible answers for this. However, first the disclaimer that level is a function of time played and not skill.
1. Matchmaking ratings carried over from beta to live. That "level 1" could have logged hundreds of games in beta and reached your skill level.
2. Your rating is close to the seed value for new players, so they are included in your search range. That is, if the seed value is 1500 and you're 1500 after 500 games, it doesn't make you a bad player, it makes you average.
3. The level 1 is part of a stack.
Now, some questions I don't have the answers to:
1) How are outliers (very low skill, very high skill) handled? Are they prioritized in the queue because their potential player pool is smaller?
2) How does the matchmaker handle backfilling? Are live games with open slots prioritized to be filled regardless of player skill?
3) Do players actually have two MMRs, one solo and one group?
4) Is the assumption of individually-applied change in MMR correct, or is it applied communally (calculated by comparing the average MMRs of both teams and applying the result equally for all players)?