Fact check: Are matches made even?

Share

A popular belief in the community is FIFA contains some kind of logic, which makes FUT matches more even. As we will show below, the evidence tells a different story.

What people believe

In the appetizer, I wrote that it is a popular belief that EA manipulates FUT matches to make them more even. However, this isn’t quite the case.

In fact, it is more accurate to say that there are a multitude of different beliefs, where the common denominator is that EA (somehow) secretly intervenes (in some way) in order to make matches more even (in some sense) in order to pursue (some sort of) commercial motive.

When you look closer at the statements that people make about matches being made even, you will find a great deal of disagreement about what is going on: Some people believe that match results are predetermined [1], while others believe that matches have a predefined bias [2]. A third group of people believe that the game will set a bias dynamically depending on the progression of the match  [3], [4], [5]. And then, a fourth group of people believes that the game will attempt to equalize all squad differences or perhaps even make the lesser team win [6], [7]. There are probably more variations to be found.

It’s difficult to address all these different claims in one post, so going forward, we will narrow the scope to the general idea that EA is making FUT online matches more even than they otherwise would be.

So, is there any truth to it?

What the evidence tells us

What evidence supports the beliefs that I presented in the previous section? As it turns out, close to nothing. I believe that I have analyzed every single piece of evidence allegedly supporting the claim that match leveling exists (some examples [1] [2] [3] [4] [5] [6]). Till this day, I’m still to come by anything, which by normal, scientific standards would qualify as evidence.

At the same time, an increasing amount of compelling evidence supports the counter claim: That match leveling a myth. Let’s a take a more thorough look at that evidence below.

Exhibit A: Skilled players don’t just get matched up against inferior opponents

The premise behind leveling matches has to be that matches wouldn’t be level without this effort. The problem with that premise is that matches between players with very different skills are quite rare: Experienced players with good win ratios just don’t come up against complete beginners.

In reality, most matches are relatively even from the start due to ELO matchmaking. ELO matchmaking means that the game will attempt to create matches, where the involved players have a similar chance of winning based on their past results. This doesn’t necessarily imply that all matches turn out as even in terms of result, but it should definitely lead to a state where most players win a fair percentage of their matches.

So, the premise behind the match leveling belief is false: Matches are not unequal from the start and they don’t need to be fixed to become reasonably equal.

Average rating of the opponents of 2 x 110 base players.

Exhibit B: Better players don’t report more manipulation

One of the main causes for these complaints is that the game sometimes evidently screws you over. We all know the feeling of conceding a stupid goals, because two defenders tackled each other, an open chance being missed horribly or a keeper letting the ball pass through – not between! – his legs. But do these things happen, because EA wants to keep matches tight?

If they do, we would expect them to happen more often to more experienced and more successful players.

However, two independent surveys have tested that assumption. Both came to the same conclusion: Better players do not experience more ‘manipulation’. In both surveys, people were asked to provide details about their experience level and skill level together with information about how often they felt that they were subject to, what they thought was, scripting, handicapping or momentum (here, manipulation). As it turns out, less skilled and less experienced players felt just as exposed to these things as more successful / more experienced players.

More experienced players do not report more ‘manipulation’ than less experienced players

So, while the game definitely screws you over, the “screwing over” doesn’t fit the pattern that we would have found if it was put in on purpose in order to give the lesser player a better chance. It does however fit well with the pattern we would expect to find, if these things just happened randomly.

Exhibit C: FUT matches aren’t even

If matches were made even by a dynamic or static bias, we would expect to see this reflected in the results.

The score line is ultimately a quite valid measure of equality in a football match. What’s the more even result: 0-0 or 9-1? Under normal circumstances, a match ending 0-0 or 2-3 would be considered even, whereas a match ending 0-3 or 5-1 would be considered less even.

So, exactly how even are FUT matches?

In the table below, we show the percentage of matches ending with various goal difference between 0 and >=5 goals. It covers FUT online singles, FUT seasons and the Big five European leagues.

Goal difference FUT 15 online singles
FUT 16 seasons Top 5 leagues
0 (draw) n/a 18 % 26%
1 46% 34 % 39%
2 24% 20 % 22%
3 12% 20 % 8%
4 10% 5 % 3%
>=5 8% 3 % 2%

(Source: EA’s Web app, game data section. Online singles data were collected during FUT 14, whereas Seasons data were collected during FUT 16. )

FUT is considerably less even than the real football matches we compare against, and the majority of FUT matches are not even by absolute numbers.

Now, the comparison against real life ultimately doesn’t rule out the possibility that EA could be intervening in order to raise the degree of evenness from an even lower level.

In order to test whether this is a possibility, we need other methods:

Under natural circumstances – that is if no scripting exists – the probability of drawing equals the probability that the involved players score the same number of goals. The probability of two teams scoring the same number of goals depends on the number of goals being scored in an average match as illustrated by the chart below:

Actual goal ratios and draw ratios for real leagues. FUT is the red dot.
Actual goal ratios and draw ratios for real leagues. FUT is the red dot.

If there was a match leveling mechanism in the game, equalizing or lead-minimizing goals would be more likely to happen than other goals. So, if there was a mechanism pulling matches in a more even direction by making people concede goals and miss chances in “strategic” situations, this would be reflected in the result statistics as an unnaturally large amount of draws and narrow wins than could be explained by pure chance.

As it however turns out, FUT’s actual draw ratio is almost exactly what it should be. In this article, we identify FUT’s natural draw ratio as 19 %, which is slightly higher than it’s actual draw ratio. Hence, there isn’t a residual of even results, which can’t be explained by natural causes.

Wrapping up the evidence

When you look for evidence of a match leveling mechanism in places where you would expect to find it, you don’t find anything:

  • FUT doesn’t have a surplus of even matches.
  • Better players don’t experience more “manipulation” than lesser players.

This is not really surprising, because when you match players based on their skill level, you don’t need to make matches even: They already are even!

I’m fully aware that people occasionally lose against opponents, whom they would beat 9 out of 10 times. But that just doesn’t indicate foul play. Let’s not forget that football is a highly random game. There is a reason why Barcelona doesn’t win every single match. If FIFA was 100 % about skill, the better player would win every time, but as soon as you allow a bit of randomness, the lesser player will pull a lucky win now and then.