Fact check: Do prime icons create fewer goals?

Share

Do prime icons create fewer goals than their lower rated version? According to one of Arlington69’s many posts on momentum and handicapping, it does. But is there really a basis for this claim?

The Claim

FUTBIN’s Player Game Performance section delivers the data behind Arlington69’s claim. In the table below, he compares the prime, mid and base versions of a number of icons.

When looking at goal creation, i.e. scoring and assists combined, he found that the lower rated version in many cases performs better than the prime version. In the post where he summarizes why he believes in momentum, he writes that:

“The highest rated icons often did not perform as well as the lower rated in fact only 3 of 11 prime icon strikers created more goals than their lower rated versions. One possible explanation for this is handicapping stopping the highest rating Icons performing to their potential.”
(– Post on Reddit)

Before we even get to handicapping and momentum, the first question we need to ask ourselves is: Does this statistic really show that prime icons perform worse?

Criticism

A recurring problem in Arlington69’s posts is that he succumbs to cherry picking. Arlington69 already knows the conclusion up front, and attributes no significance to data that doesn’t fit in. When you look at the data he presents in this case, it’s remarkable how much of it that actually doesn’t fit with his conclusion.

Arlington69 notes that only 3 in 11 prime icons in his comparison perform better than the lower rated versions. He argues that this could suggest that handicapping exists. But he makes no attempt to explain how those three prime icons that did outperform their earlier versions possibly could fit into that theory. Additionally, he for some unknown reason ignores that his sample actually consists of 15 players, among which 6 prime editions outperform the lower rated versions. He also ignores that 11 of 15 prime icons outperform minimum one of the earlier versions.

Another problem is that he fails to recognize the limitations of FUTBIN’s player game performance data. We have uses PGP data previously [1][2][3][4][5], and are therefore painfully aware of some of the limitations.

First, although Arlington69’s samples consist of several thousand matches, they are far too small in terms of the number of owners. To demonstrate this, we looked up the performance data for some of his icons today, June 30th: Prime Del Piero’s “goals created” is 1.18 (Arlington69: 1.12). Mid Owen’s “goals created” is 1.43 (Arlington69: 1.75). Base Shevchenko’s “goals created” is 1.42 (Arlington69: 1.35).  In other words, the measurements put forward by Arlington69 are inaccurate. It’s most likely a pure coincidence that he found prime Owen to perform worse than base Owen, whereas prime Shearer was found to outperform his earlier versions.

Second, Arlington69 fails to consider that multiple factors affect the performance. What he basically is trying to do here is to measure how a card’s basic attributes (stats) influence the performance. In order to do that, he needs to isolate the effects caused by stats from the effects caused by other influencing factors, not least human skill and team-mate performance.

To neutralize the effects caused by human skill, you need to ensure that the owners of the compared cards are evenly skilled on average. This is difficult when comparing icons, because you inevitably will be looking at a small sample of cards, which have had few different owners. Additionally, the prices vary considerably, meaning that different editions may appeal to different owner segments. It’s difficult to support the assertion that players owning a card worth 2 million coins have the same average skill level as the players who have owned a 20,000 coins card.

Neutralizing the effects of team-mate performance is per definition impossible no matter whether you are looking at shots or assists. But the problem is all other things equal more prevalent with assists, where your player needs someone else to score a goal before he can celebrate. As Arlington69 combines shots and assists into one performance measure, he ends up with a pretty unreliable, combined measure of how the player’s own stats actually influence his performance.

Conclusion

In summary, Arlington69’s data doesn’t confirm that prime icons perform worse than lower rated editions. This is not a surprise though.

First and foremost, we have conducted numerous studies on player performance, and all of them have shown that players with better, relevant stats score more goals.

Second, the claim made by Arlington69 is nonsense! He basically asserts that EA has reversed the effect of stats, effectively making it an advantage to use cards with low stats. Why would EA do that? It won’t make matches more even – just turn the advantage upside down. And it removes the incentive to buy packs.

Another notable detail about Arlington69’s post, “Why I believe in momentum (SHM)”, is that he points to six different pieces of evidence suggesting that momentum exists. The evidence discussed in this article is one of those pieces, despite the fact that it – according to his own interpretation – supports the idea that better teams are handicapped! Although Arlington69 makes no attempt to define handicapping and momentum, it remains clear from his writings that he distinguishes between them:

“Since I started playing again in Fifa 17 it seemed clear to me that the game would swing in favour of one player or another and usually the player losing or with the worse team.”
(– Quote from Reddit post)

Handicapping refers to the idea that EA favors the worse team, whereas momentum refers to the idea that there is a “trend for the game to favour the losing team” to use Arlington69’s own words.

However, handicapping and momentum aren’t two sides of the same coin. The handicapping concept, Arlington69 refers to, would always favor the worse team no matter who is winning, whereas momentum would favor the losing side no matter who has the better team. Hence, these two concepts are incompatible. You can’t have both. And yet, Arlington69 is using evidence allegedly supporting handicapping to argue why he believes in momentum. This is not exactly the kind of stuff that will bring you the Nobel prize…

%d bloggers like this: