Fact check: Are matches made even?

Share

A popular belief in the community is FIFA contains some kind of logic, which makes FUT matches more even. But as we will show below, the evidence tells a different story.

What people believe

A lot of people subscribe to the broad range of beliefs about EA (somehow) secretly intervening (in some way) in order to make matches more even (in some sense) in order to pursue (some sort of) commercial motive.

When you look closer at the statements that people make about matches being made even, you will find a great deal of disagreement: Some people believe that match results are predetermined [1]. Others believe that matches have a predefined bias [2]. A third group of people believe that the game will intervene dynamically depending on the progression of the match [3], [4], [5]. And then, a fourth group of people believes that the game will attempt to equalize all squad differences or perhaps make the lesser team win [6], [7]. There are probably more variations to be found.

It’s nearly impossible to address all different claims in one post, so going forward, we will narrow the scope to the idea that EA makes FUT online matches result-wise more even than they otherwise would be through dynamic intervention in favor of the trailing side.

What the evidence tells us

Does any evidence support the belief that EA makes matches more even? I believe I have analyzed every single piece of evidence allegedly supporting the claim that match leveling exists (some examples [1] [2] [3] [4] [5] [6]). Till this day, I’m still to come across any information, which by normal, scientific standards would qualify as evidence.

At the same time, an increasing amount of compelling evidence supports the opposite conclusion. In the next sections, I will present four different pieces of hard, factual evidence, which altogether leads to the same conclusion: That EA isn’t making matches even.

Exhibit A: Skilled players don’t just get matched up against inferior opponents

A key premise behind the leveling matches belief has to be that matches wouldn’t be level without this effort. But obvious problem with that premise is that matches between players with very different skills are quite rare.

Experienced players with good win ratios usually don’t come up against complete beginners due to that FIFA uses Elo matchmaking.

Average rating of the opponents of 2 x 110 base-players.

Elo matchmaking means that the game will attempt to create matches, where both the involved players have a fair chance of winning based on their past results. This doesn’t necessarily imply that all matches turn out as even in terms of result, but it should definitely lead to a state where most players win a fair percentage of their matches in the longer run.

Thus, the first problem with the theory that matches are being made even is that it is based on a false premise: Namely that matches wouldn’t be even without it. So, the situation is that we are supposed to believe that EA intervenes to make matches even when there is no need to do so.

Exhibit B: Better players don’t report more manipulation

The game sometimes evidently screws you over, and sometimes that causes you to lose or draw. We all know the feeling of conceding a stupid goal, because two defenders tackled each other, an open chance being missed horribly or a keeper letting the ball pass between or through his legs. But do such things happen, because EA wants to keep matches tight?

If this screwing-us-over was built in to the game in order to keep matches tight, we would expect it to happens more often to experienced and successful players than to less successful and less experienced players. Most of the time, there should be little reason to screw the lesser side over in order to make a match even.

So, do more experienced players get screwed over more often? We tested this in two, independent studies. Both studies arrived at the same conclusion: Better players do not experience more ‘manipulation’. In both surveys, people were asked to provide details about their experience level and skill level together with information about how often they felt that they were subject to, what they thought was, scripting, handicapping or momentum (here, manipulation). As it turns out, less skilled and less experienced players felt just as exposed to these things as more successful / more experienced players.

More experienced players do not report more ‘manipulation’ than less experienced players

So, while it might be true that the game screws you over, the “screwing over” doesn’t occur in the pattern we would expect if it was put in intentionally with the purpose of helping the lesser player. It does however fit well with the pattern we would expect to find, if these things just happened randomly.

Exhibit C: FUT matches aren’t even

If matches were made even by a dynamic or static bias, we would expect to see this reflected in the results. If EA is leveling matches, we would expect to see fewer 5-0’s and more 2-1’s than there otherwise would have been.  The scoreline is a valid measure of equality in a football match.

So, exactly how even are FUT matches in terms of scoreline?

In the table below, we calculate the percentage of matches ending with various goal difference between 0 and >=5 goals. The matches in the table came froma data set of +3000 matches played by 110 different players. The table shows data from FUT online singles, FUT seasons and the Big five European leagues, i.e. real life football.

Goal difference FUT 15 online singles
FUT 16 seasons Big 5 leagues
0 (draw) n/a 18 % 26%
1 46% 34 % 39%
2 24% 20 % 22%
3 12% 20 % 8%
4 10% 5 % 3%
>=5 8% 3 % 2%

(Source: EA’s Web app, game data section. Online singles data were collected during FUT 14, whereas Seasons data were collected during FUT 16. )

It is clearly evident that FUT matches are considerably less even than real football matches, and that the majority of FUT matches aren’t even by normal measures.

Of course it could be argued that the comparison against real life ultimately doesn’t rule out the theoretical possibility that EA could be intervening in order to raise the degree of evenness from an even lower level.

We can however rule that theoretical possibility out by looking at the percentage of ties.

Under natural circumstances – that is if no match leveling exists – the probability of a match ending in a tie equals the probability that the involved players score the same number of goals. Long sentence, simple point.

The probability of two teams scoring the same number of goals normally depends on the number of goals being scored in an average match. You perhaps noticed that high scoring sports like basketball and handball contain fewer draws than football. In the same way, football leagues with more goals have fewer draws than football leagues with fewer goals as illustrated by the chart below:

Actual goal ratios and draw ratios for real leagues. FUT is the red dot.
Actual goal ratios and draw ratios for real leagues. FUT is the red dot.

If there was a match leveling mechanism in the game, equalizing or lead-minimizing goals would be more likely to happen than lead-increasing goals. So, if there was a leveling mechanism pulling matches in a more even direction by making people concede goals and miss chances in “strategic” situations, this would be reflected in the result statistics as an unnaturally large amount of draws and narrow wins than couldn’t be explained by pure coincidence.

In this article, we identify FUT’s natural (expected) draw ratio as 19 %, which in fact is slightly higher than its actual draw ratio.

Hence, FUT matches aren’t particularly even and importantly, there isn’t a residual of even results, which can’t be explained by natural causes.

Exhibit D: Undeserved wins are very rare

Sometimes you lose even though you felt that you dominated the match. Your after-match stats confirm that you dominated the match on shots, possession and passing accuracy. But how often does the dominating player actually lose?

We checked this out by looking at a dataset of 1,100 matches played during FUT 16. All of them were FUT seasons matches. We wanted to know the percentage of matches where the losing party dominated on all three performance metrics (passing, possession and shooting) and had a better historic track record.

We found that in 8 % of all matches, the winning party didn’t dominate on any of the performance metrics. But in roughly half of those matches, the winning party still had the better historic performance track record, meaning that he still could have been the better player.

That leaves us down to a residual of 4 % of all the matches (40 in total) where (a) the player with the worse overall track record and (b) worst performance in the match won.

So, did EA intervene in 4 % of the matches then in order to help out a new or bad player?

A deep dive into those 40 matches shows that in 33 cases, the winning player had completed division 1, 2 or 3 as his best. Only in a 2 matches did the winning player reside below division 5, but in both cases, so was his opponent! So, even the 40 matches where the apparent lesser player won still don’t fit with the narrative that these results occurred because EA intervened in those matches to help out bad players.

When we look at the large dataset consisting of more than 1000 matches, we didn’y find any matches where the in disputably better player lost.

Wrapping up the evidence

When you look for evidence of a match leveling mechanism in places where you would expect to find it, you don’t find anything:

  • FUT doesn’t have a surplus of even matches.
  • Better players don’t experience more “manipulation” than lesser players.
  • Better players don’t lose more matches than they ought to.

This is not really surprising, because when you match players based on their skill level, you don’t need to make matches even: They already are even!

I’m fully aware that people occasionally lose against opponents, whom they would beat 9 out of 10 times. But that just doesn’t indicate foul play. Let’s not forget that football is a highly random game. There is a reason why Barcelona doesn’t win every single match. If FIFA was 100 % about skill, the better player would win every time, but as soon as you allow a bit of randomness, the lesser player will pull a lucky win now and then.

%d bloggers like this: