Understanding matchmaking in FUT is a key issue. In an earlier article, we confirmed that FUT seasons matches you against opponents with similar skills, in effect what is known as ELO matchmaking. Based on those findings, an intriguing question came up: If our opponents are as skilled as ourselves, then how come that some players win 60 % of their matches, while others only win 20 %? In this article, we try to come up with a qualified guess to answer that question.
To confirm that that FUT seasons uses ELO matchmaking, we used our dataset covering 218 random players and their 10 latest opponents. For each of these ~2,200 players, we calculated a skill rating consisting of win ratio, best completed division and goal difference per match. In the chart below, we plotted each of our 218 base players, having the “average skill level of the player’s 10 latest opponents” as the Y-coordinate and the base player’s own skill level as the X-coordinate.
The chart confirms the existence of a relationship (correlation) between the player’s own rating and the average ratings of his 10 latest opponents. Speaking statistics, this implies that you are more likely to come up against an opponent who is on par with you skill-wise than the opposite.
To further support his conclusion, take a look at the two charts below, where we have created random matches within the same player population that forms the basis of the actual data set used above.
- Yellow chart: We matched players completely randomly. For each base player, we picked 10 random opponents within the full population of 3200 opponents.
- Red chart: We picked three, random opponents per match and chose the the opponent’s whose skill rating came closest to the skill rating of our base player’s own skill rating.
It is clearly visible is that the red chart comes significantly closer to the actual matchmaking results (the blue chart above) than the yellow chart.
The flat S
Although the game has a preference for opponents with similar skill levels, not all players have the same probability of getting matched against an opponent who is below and above them in terms of skill.
When we look at the lowest rated players (to the far left), their average opponents are somewhat superior, whereas the average opponents of the better players to the right were slightly inferior to the players themselves. If we draw a trend line (red) through the data set, we see a “flat S”-shaped curve rather than the straight, 45 degree line, which we would have seen if everyone’s average opponents had the same rating as themselves.
How come, that bad players on average get opponents who are slightly superior to themselves, whereas good players get opponents who are slightly inferior to themselves?
The bell shaped curve
Less successful players get a higher percentage of superior opponents, while more successful players get a higher percentage of inferior opponents than the average player. Our best candidate for an explanation for that has to do with the way, players are distributed skill-wise.
As would have been the case if we were measuring height, shoe size or intelligence, the majority of players are average skilled with a small minority having extraordinarily good skills and an approximately similar sized group having extraordinarily poor skills. In statistical terms, we call this a normal distribution. When plotted into a chart, it shows up as a bell shaped curve. Below, I have plotted two different, skill-related stats: Win-ratio (orange) and the combined Skill rating (blue).
The consequence of this distribution is that players with different skill profiles face different probabilities of pulling superior and inferior opponents.
If we for a second forget about ELO matchmaking and assume that matchmaking is completely random, a player with a win ratio of 30 % would have 97 % chance of pulling an opponent with a higher win ratio, whereas a player with a win ratio of 35 % would have 14 % chance of pulling an opponent with a lower win ratio.
The fact that FUT uses ELO matchmaking doesn’t change the basic fact that you only can get matched against opponents who (a) have decent connectivity to you and (b) are searching for a match at the same time. Hence, at the end of the day, the matchmaking algoritm will have to pick the best available match in terms of ELO ranking among the alternatives available at the given time.
Even though the game will pick the best available match, the best available match for a bad player is more likely to be a superior player than an inferior player, which probably is why our blue curve has it’s flat S-shape.
The number of alternatives available
You might have noticed that even though our ELO simulation (the red chart) had some resemblance with the actual matchmaking results (the blue chart), the curves weren’t exactly the same: The red curve appears “tighter packed”, which translated into human language means that our simulation is too effective in finding opponents with a similar skill level when compared to what is possible in the real world.
Considering that approximately 5,000 FIFA matches are started every minute, one might have expected a success rate when looking for similarly skilled opponents. So, how come that the actual matchmaking results appear as random as they do?
We can come up with minimum three, possible reasons.
- One is that our ELO rating system isn’t the same system that EA is using, meaning that our assessment of the players’ skill levels is somewhat off. In essence, we don’t know how accurate our skill ranking system is. Hence, although the actual matches appear more random through the binoculars that we build for ourselves, we don’t know whether they appear equally random through EA’s presumably even more accurate binoculars.
- The second possibility is statistical accuracy. After all, we calculate the average opponent’s skill level based on just 10 opponents. The inevitable consequence is that we will have more outliers, as just one odd opponent will skew the average considerably.
- The third possible reason is that the game, in spite of the large number of matches being started, perhaps doesn’t have that many alternatives available when trying to find a match due to connectivity restrictions. Below, we ran the same simulation with 3 respectively 2 opponent alternatives per match. It is clearly visible that the more alternatives available, the more even matches will you get. The resemblance between the actual matchmaking (the blue curve) and the simulation with only two alternatives available is clearly bigger than when we allow the simulation to chose between three opponents.