Now and then, we come across small, home brewed experiments designed to provide evidence in support of scripting or handicapping. We decided to do a small series of articles to take a closer look at some of these experiments. In this post, we take a critical look at a small experiment made by author Silvio Teixeira and presented in this article on FIFAUTEAM in late 2014.
Texieira discusses and tests (well, sort of) various claims with regards to manipulation in FIFA. He ends up concluding that players with better squads aren’t handicapped, whereas better players are.
But did Teixeira really produce evidence allowing him to conclude that the game without a doubt deliberately makes you lose when you didn’t deserve to? The answer is, not surprisingly, a clear and loud no.
When the label pseudo science would be an exaggeration
Although people commenting on Teixeira’s article clearly attribute some sort of factual validity to it, it isn’t science by any measures.
Teixeira starts out discussing the claim that players with weaker squads get helped by the game. He tests the claim in a small experiment where he plays some matches against a friend, whom according to Teixeira usually loses against 8 out of 10 times or so. First, they play 6 matches where Teixeira has a superior squad. He ends up winning 3, drawing 2 and losing 1 match. Then, they swap teams, and Teixeira loses 5 out of 6. This leads him to conclude that players with better squads aren’t handicapped.
Even though the conclusion is consistent with our own research, the sample size and the method in general is clearly insufficient to conclude anything from, which is brilliantly illustrated by the fact that similar experiments have reached the exact opposite conclusion using roughly the same approach.
They make you lose for some reason…
Following his rebuttal of squad handicapping, Teixeira moves on to discuss the broader claim that the game makes you lose on purpose. The following quote wraps up his conclusion well as the data which (supposedly) supports it:
“Now, if you’re wondering if there’s in fact some force that influences your game play making you lose when you probably didn’t deserve to, the answer is YES, without a doubt. It’s just not related to squad rating. How did I come to this conclusion? It was simple, yet pretty tough to settle. On certain matches where I felt like there was some disadvantage going on I’d usually lose 3 or 4-0. In cases like this, I’d explain my intentions to the opponent and then add them for a rematch using the same teams, except that on a friendly. On both rematch situations where I managed to win, one of which I showed absolute superiority, it didn’t even seem like I was playing against the same person. It happened the other way around, too. I won a few but then I wouldn’t be able to make a good appearance on friendly rematches, and the opponent and I would even try squad rating variations. Nothing changed.”
So, what this boils down to is that Teixeira experienced that he could win and lose by relatively comfortable margins against the same opponents. Therefore, he concludes, the game must be manipulated.
There are several things wrong with this conclusion.
Inconsistency is normal in football
First of all, Teixeira’s argument is a false cause fallacy. You can’t conclude that manipulation indeed was the cause behind an event, because it theoretically could be the cause behind it. Even though Teixeira may have had difficulties imagining other possible causes for the inconsistency he experienced, there definitely are other possible causes. The first thing that ought to leap into mind is coincidence. Football is the biggest betting sport in the world for a reason. It probably wouldn’t be if it wasn’t for a staggering amount of randomness. In reality, no teams are as consistent as Teixeira expects them to be.
In football it’s a perfectly normal thing to win and lose against the exact same opponent during the same season. Even a highly consistent teams like FC Barcelona only manages to win both matches against the same opponent in 2/3 of the cases.
| Results against same opponent
(CL + League)
|Consistent: Won, drew or lost both||16||6|
|Inconsistent: Won one, lost one||5||5|
|Inconsistent: Won one, drew one||3||6|
|Inconsistent: Lost one, drew one||0||2|
What causes inconsistency in real football? It’s difficult to pinpoint one reason, but broadly speaking, the entire setup around a football match (a ball, a grass pitch, weather, 22 humans, few goals etc.) forms the perfect conditions for some highly unpredictable events to unfold. Although FIFA is a computer game, many of the same ingredients are present there as well. The primary difference between FIFA and real football in this respect is perhaps that FIFA’s goal frequency is significantly higher, meaning that you more easily win or lose by a large margins.
Hence, there definitely are other possible reasons for the inconsistencies experienced by Teixeira.
Another problem with Teixeira’s argument is the lack of a coherent causal theory. Teixeira concludes that EA makes you lose on purpose, but he fails to explain what their rationale could be. The closest we get to a rationale are the two quotes below:
“I’m sure there’s some mechanic behind determining if you’re getting an advantage or disadvantage.
EA understands that this is the best thing to be done.”
Essentially, Teixeira is applying reverse logic here: They did it, and therefore there must have been some reason for them to do it. In reality, they for all we know didn’t do it, because it would make absolutely no sense.
Why we should discuss articles like these
One of the purposes of discussing an article like Texieira’s is to demonstrate how easy it is to create an experiment and present it in a way, which at a glance appears to prove something, even though it’s nonsense from one end to another. I wouldn’t even call this article pseudo science, because it doesn’t give off the impression of presenting anything other than opinions. Yet, I think it’s fair to evaluate such articles from a scientific view point, as a lot of people clearly accept them as fact.