Fact check: Are matches made even?

Share

A popular belief in the community is FIFA contains some kind of logic, which makes FUT matches more even. But as we will show below, the evidence tells a different story.

What people believe

A lot of people subscribe to the broad range of beliefs about EA (somehow) secretly intervening (in some way) in order to make matches more even (in some sense) in order to pursue (some sort of) commercial motive.

When you look closer at the statements that people make about matches being made even, you will find a great deal of disagreement about what is going on: Some people believe that match results are predetermined [1], while others believe that matches have a predefined bias [2]. A third group of people believe that the game will set a bias dynamically depending on the progression of the match [3], [4], [5]. And then, a fourth group of people believes that the game will attempt to equalize all squad differences or perhaps make the lesser team win [6], [7]. There are probably more variations to be found.

It’s difficult to address all these different claims in one post, so going forward, we will narrow the scope to the perceptionthat EA is making FUT online matches more even than they otherwise would be.

What the evidence tells us

What evidence supports the belief that EA makes matches more even? As it turns out, nothing. I have analyzed every single piece of evidence allegedly supporting the claim that match leveling exists (some examples [1] [2] [3] [4] [5] [6]). Till this day, I’m still to come across any information, which by normal, scientific standards would qualify as evidence.

At the same time, an increasing amount of compelling evidence supports the opposite conclusion. In the next sections, I will present four different pieces of hard, factual evidence, which altogether leads to the same conclusion: That EA isn’t making matches even.

Exhibit A: Skilled players don’t just get matched up against inferior opponents

A key premise behind the leveling matches belief has to be that matches wouldn’t be level without this effort. But obvious problem with that premise is that matches between players with very different skills are quite rare:

Experienced players with good win ratios usually don’t come up against complete beginners due to that FIFA uses Elo matchmaking. Elo matchmaking means that the game will attempt to create matches, where both the involved players have a fair chance of winning based on their past results. This doesn’t necessarily imply that all matches turn out as even in terms of result, but it should definitely lead to a state where most players win a fair percentage of their matches in the longer run.

Thus, the first problem with the match leveling belief is that it is based on a false premise: Namely that matches wouldn’t be even without it. That is definitely not the case.

Average rating of the opponents of 2 x 110 base players.

Exhibit B: Better players don’t report more manipulation

The game sometimes evidently screws you over, and sometimes that causes you to lose or draw. We all know the feeling of conceding a stupid goal, because two defenders tackled each other, an open chance being missed horribly or a keeper letting the ball pass between or through (!) his legs. But do such things happen, because EA wants to keep matches tight?

If this screwing us over was build in to the game in order to keep matches tight, we would expect it to happens more often to experienced and successful players than to less successful and less experienced players.

This was tested in through two, independent studies. Both studies arrived at the same conclusion: Better players do not experience more ‘manipulation’. In both surveys, people were asked to provide details about their experience level and skill level together with information about how often they felt that they were subject to, what they thought was, scripting, handicapping or momentum (here, manipulation). As it turns out, less skilled and less experienced players felt just as exposed to these things as more successful / more experienced players.

More experienced players do not report more ‘manipulation’ than less experienced players

So, while the game definitely screws you over, the “screwing over” doesn’t fit the pattern that we would have found if it was put in on purpose in order to give the lesser player a better chance. It does however fit well with the pattern we would expect to find, if these things just happened randomly.

Exhibit C: FUT matches aren’t even

If matches were made even by a dynamic or static bias, we would expect to see this reflected in the results.

The score line is ultimately a quite valid measure of equality in a football match. What’s the more even result: 0-0 or 9-1? Under normal circumstances, matches ending 0-0 or 2-3 would be considered more even than matches ending 0-3 or 5-1.

So, exactly how even are FUT matches?

In the table below, we show the percentage of matches ending with various goal difference between 0 and >=5 goals. The tables shows data from FUT online singles, FUT seasons and the Big five European leagues, i.e. real life football.

Goal difference FUT 15 online singles
FUT 16 seasons Big 5 leagues
0 (draw) n/a 18 % 26%
1 46% 34 % 39%
2 24% 20 % 22%
3 12% 20 % 8%
4 10% 5 % 3%
>=5 8% 3 % 2%

(Source: EA’s Web app, game data section. Online singles data were collected during FUT 14, whereas Seasons data were collected during FUT 16. )

The data shows that FUT matches are considerably less even than real football matches, and that the majority of FUT matches aren’t even by normal measures.

Of course it could be argued that the comparison against real life ultimately doesn’t rule out the possibility that EA could be intervening in order to raise the degree of evenness from an even lower level.

In order to test whether this is a possibility, we need other methods:

Under natural circumstances – that is if no match leveling exists – the probability of a match ending in a tie equals the probability that the involved players score the same number of goals. Long sentence, simple point.

The probability of two teams scoring the same number of goals normally depends on the number of goals being scored in an average match. That’s why high scoring sports like basketball and handball contain fewer draws than football. Also, football leagues with more goals have fewer draws than football leagues with fewer goals as illustrated by the chart below:

Actual goal ratios and draw ratios for real leagues. FUT is the red dot.
Actual goal ratios and draw ratios for real leagues. FUT is the red dot.

If there was a match leveling mechanism in the game, equalizing or lead-minimizing goals would be more likely to happen than other goals. So, if there was a leveling mechanism pulling matches in a more even direction by making people concede goals and miss chances in “strategic” situations, this would be reflected in the result statistics as an unnaturally large amount of draws and narrow wins than couldn’t be explained by pure coincidence.

As it however turns out, FUT’s actual draw ratio is almost exactly what it should be, given it’s goal ratio. In this article, we identify FUT’s natural (expected) draw ratio as 19 %, which in fact is slightly higher than it’s actual draw ratio.

Hence, FUT matches aren’t particularly even and importantly, there isn’t a residual of even results, which can’t be explained by natural causes.

Exhibit D: Undeserved wins are very rare

Sometimes you lose even though you felt that you dominated the match. Your after-match stats may also confirm that you dominated the match on shots, possession and passing accuracy. But how often does the better player actually lose?

We checked this out by looking at a dataset of 1,100 matches played during FUT 16. All of them were FUT seasons matches. We wanted to know the percentage of matches where the losing party dominated on all three performance metrics (passing, possession and shooting) and had a better historic track record.

We found that in 8 % of all matches, the losing party didn’t dominated on any of the performance metrics.

But in roughly half of those matches, the winning party still had the better historic performance track record.

That left us down to a residual of 4 % of all the matches (40 in total) where (a) the player with the worse overall track record and (b) worst performance in the match won.

So, did EA intervene in 4 % of the matches then in order to help out a new or bad player?

A deep dive into those 40 matches shows that in 33 cases, the winning player had completed division 1, 2 or 3 as his best. Only in a 2 matches did the winning player reside below division 5, but in both cases, so was his opponent! So, even the 40 matches where the apparent lesser player won still don’t fit with the narrative that these results occurred because EA intervened in those matches to help out bad players.

Wrapping up the evidence

When you look for evidence of a match leveling mechanism in places where you would expect to find it, you don’t find anything:

  • FUT doesn’t have a surplus of even matches.
  • Better players don’t experience more “manipulation” than lesser players.
  • Better players don’t lose more matches than they ought to.

This is not really surprising, because when you match players based on their skill level, you don’t need to make matches even: They already are even!

I’m fully aware that people occasionally lose against opponents, whom they would beat 9 out of 10 times. But that just doesn’t indicate foul play. Let’s not forget that football is a highly random game. There is a reason why Barcelona doesn’t win every single match. If FIFA was 100 % about skill, the better player would win every time, but as soon as you allow a bit of randomness, the lesser player will pull a lucky win now and then.

%d bloggers like this: