Arlington69 has written yet another post here he claim to have evidence supporting the presence of a momentum feature. The basis of the claim is an apparently suspicious observation: In matches where he went from 2-2 to 3-2, he and his opponent tend to perform evenly from 3-2 and on wards. But in matches that went from 3-1 to 2-2, the previously trailing player tends to perform better after 3-2.
Translated into human language, it seems that it becomes harder to score additional goals if you went from 3-1 to 3-2 rather than 2-2 to 3-2.
Did Arlington69 finally manage to prove that momentum exists? We decided to give his logic a little service check.
Below, we see the important parts of Arlington69’s aforementioned statistic. The table covers all matches where he at some point was leading 3-2.
|n||Scenario leading to 3-2||Goals (own) scored after 3-2||Goals (opponent) scored after 3-2|
|26||3-1 –> 3-2||0||11|
|102||2-2 –> 3-2||30||30|
We see that in 26 cases, he went from 3-1 to 3-2. In those cases, his opponents continued to score 11 goals after 3-2 while he scored 0.
And then we have another 102 cases, where he went from 2-2 to 3-2. In those cases, both Arlington69 and his opponents scored a total of 30 goals after 3-2.
It definitely looks as if it became harder for Arlington69 to score goals at the scoreline 3-2 when he went from 3-1 to 3-2 rather than from 2-2 to 3-2.
Based on these observations, he infers that the game has build in logic, which creates momentum swings:
“In conclusion this is more evidence that there are momentum swings within EA’s FIFA 20. I believe this is coded into the game”
We have no reason to believe that the data are incorrect. Rather, the question that begs for attention here is whether we actually can conclude that momentum is likely to exist based on these observations.
The momentum-mechanism that Arlington69 believes is present, is a feature which allegedly should make matches more even:
“[I]t seemed clear to me that the game would swing in favour of one player or another and usually the player losing or with the worse team.”
(– Quote from “Why I believe in momentum” on Reddit)
But isn’t there an obvious problem here?
The very first thing we notice when we look at his data is that the majority of matches – 4 in 5 – in fact became less even, as they went from 2-2 to 3-2.
Also worth noting is a “scoreline flowchart”, which Arlington69 created for another blog post. The flow chart illustrates how often a certain score line in a match developed into a certain other scoreline. It is based on the same data used in the analysis discussed here. According to said flowchart, 2-3 or 3-2 leads only transformed into the most even result of them all – a draw – in 16 % of the cases.
So, if we look at the full deck of data put in front of us here, we find it difficult to justify the claim that these results were caused by a build in mechanism, which favors the losing player.
In fact, it would seem that there are plenty of data which points in the opposite direction of the conclusion that Arlington69 favors.
Having said that, it still would be a huge stretch to claim that any of his data, even when seen in isolation, suggests that momentum exists.
False cause fallacy
It is a fact that Arlington69 scored more goals when he went from 2-2 to 3-2 than when he went from 3-1 to 3-2. Hence, the (a) “score-line before 3-2” and the b) “goal distribution after 3-2” definitely appear to be correlated although the statistical uncertainty (small sample sizes) prevents us from concluding this with certainty.
But a quick peek on page 1 in any science book would tell you what Arlington69 clearly missed here:
The fact that two things are correlated doesn’t lead to the conclusion that they also are causally connected, i.e. in this case that it became more difficult to score at 3-2 because he came from 3-1 rather than 2-2.
Before we can conclude anything about causality, we need to effectively rule out all other possible explanations.
Arlington69 made no attempt at ruling out any other explanations. Maybe he wasn’t looking, maybe he just didn’t see it. But no matter what, there is another very possible explanation: Namely the statistical phenomenon known as regression to the mean.
Regression to the mean
Regression to the mean can be seen everywhere in life, but football has brought many examples.
One such example is a situation from the Spanish La Liga in the 2018/19 season. You may recall that Paulinho in mid January had surprised everyone by scoring an impressive 8 goals. Meanwhile, Cristiano Ronaldo was having his worst season ever as he scored only half of that; namely 4.
And then, Paulinho and Ronaldo both were hit smack in the face by regression to the mean:
When La Liga ended in May, Paulinho had increased his goal tally to just 9 goals, whereas Cristiano Ronaldo had increased it to an impressive 26 in total.
In that way, the season ended up as an average season for both players. They both regressed to the mean.
If you were to apply Arlington69’s logic here, you would compare the situation in January with the situation in May. And you would see what Arlington69 would call a momentum shift: The formerly trailing Ronaldo catching up with Paulinho and eventually surpassing him. However, it doesn’t take much football knowledge to realize that this so-called “momentum shift” is completely natural. We are clearly not witnessing a God-like force intervening in Spanish football but plain and simply that a player’s performance may start at an extreme high or low, but still eventually normalize.
Given that Arlington69’s experiment is based on his own matches, a natural question to ask is whether Arlington69 is Ronaldo or a Paulinho when it comes to FIFA. Or to put it differently: What is his “mean” and what is his “extreme”?
His scoreline flowchart provides a few hints about his skill level relative to his average opponent:
Arlington69’s opponents scored 2 % more goals than him, but he won 1 % more matches than he lost. None of the deviations are statistically significant. Hence, a fair conclusion is that Arlington69 is an average player relative to his opponents.
With that knowledge, let’s take another look at his experimental design.
Designed to fail
Arlington69’s experiment is a classic comparison of two samples.
Sample #1 consists of matches where he was leading 3-1, meaning that sample #1 contains matches where he on average was performing above his own norm at some point.
Sample #2 consists of matches where the scoreline at some point was 2-2, meaning that he at that point was performing to his own norm.
So, in essence, he picks a sample #1 which by design is prone to regression to the mean and compares it to a sample #2 which isn’t.
And when you do that, you should expect to see exactly what Arlington69 sees here: Namely that the matches in sample #1 tend to regress from above average to average, whereas the matches in sample #2 in general won’t change because they already are average.
Arlington69’s latest experiment follows in the footsteps of a long list of earlier, failed attempts to prove momentum from his side. But it is also another lesson into why people tend to believe such things in the first place.
Regression to the mean is just as inevitable as night and day. But few people have heard about it, and even fewer understand the implications. The observations that Arlington69 makes are without question very real, and numerous other players have witnessed these things, although few of them have bothered recording it in a spreadsheet. But they are not only real but also completely natural.
Yet, combined with our tendency to prefer explanations, which attribute a meaning to otherwise random events, we have everything we need for yet another good conspiracy theory.