A user called Dennisisspiderman recently claimed that AI-controlled players act more aggressively against superior squads. The claim, if true, would have significant implications. Not only for the bronze benching debate but also for the handicapping debate. Additionally, Dennisisspiderman presents some evidence which at a glance appears interesting. Given these circumstances, we decided to investigate the claim.
Dennisispiderman played a number of FUT matches against the AI. In all matches, the AI was controlling a 70 rated team whereas Dennisisspiderman was controlling teams rated 50, 71, 80 and 86. This means that the rating gap in those four matches was -20, +1, +10 and +16. He then observed how the AI-controlled players acted right after kick off at different rating gaps. His observations lead to the following conclusion:
“[L]ower rated squads result in the lower rated squad becoming more aggressive.”
(– Comment on Reddit)
So, it seems that when the squads are even (small rating gap), the AI is less aggressive. If the rating gap is large, the Ai is more aggressive. In a comment to Nepenthez, Dennisisspiderman writes the following:
“I don’t believe that things happen in the game according to some “script” (like conceding a goal because the game decided you should concede one) but I believe […] that there are certainly mechanics at play to change how players (at least CPU AI ones) play on the same difficulty levels, tactics, and instructions.”
(– Comment on Reddit)
He elaborates that statement even further in two other comments:
“I believe there’s potential there to show that the difference in ratings between two sides affects not just how aggressive one team is but also how “intelligent” or “reactive” one team is over the other.”
(– Comment on Reddit)
“At least for modes like Squad Battles team rating definitely can affect performance of one team or the other (which explains why people may have a tough time playing against lower rated squads).”
(– Comment in Reddit)
In all fairness, Dennisisspiderman doesn’t claim that his observations suggest that handicapping exists. But the last comment suggests that he believes that the game “dumbs down” your team when matched against inferior squads. Further, he wisely makes the reservation that it is uncertain whether the observations will apply to online multiplayer matches.
The empirical basis of Dennisispiderman’s claim is the four observations of how the AI applies pressure after kick off.
The following quote contains direct links to the four video clips supporting the conclusion as well as some additional comments on how AI behavior changed. The quote also contains Dennisisspiderman’s notes on what he observed in each case.
“My 50 squad vs the 70 squad. The opponent team is essentially static except for Vardy who just sort of runs back and forth while still leaving players open.
My 71 squad vs the 70 squad. The opponent strikers both move around to cut lanes but two of my teammates move around to stay open.
My 80 squad vs the 70 squad. Same result as above. Opponent strikers move around to cut lanes and my teammates try to stay open.
My 86 squad vs the 70 squad. Completely different outcome. Immediately I get pressured and the opponent wins the ball. At one point my player (Alli) doesn’t even attempt to settle a ball and when I regain possession I am again quickly pressured and lose the ball.”
Although we find Dennisisspiderman’s experiment somewhat interesting, there are a number of obvious concerns.
The most obvious concern is that Dennisisspiderman’s full experiment consists of only four observations. Based on such a small sample, it is impossible to tell whether the observed changes are causally connected to the rating gap or perhaps the product of different team mentality settings.
If Dennisisspiderman was able to reproduce the same results for example 10 times, it would be difficult to reject some sort of causal connection between AI aggressivenes and rating gap. But based on the current data, it is equally likely that additional experiments would lead to a rejection Dennisisspiderman’s hypothesis.
When the data fits the hypothesis like a foot in a glove
Another problem is what the four clips actually show. What we see is the opposing squad acting with similar aggressivenes at rating gaps -20, +1 and +10, whereas the aggressiveness increases notably at +16. How well does those observations actually fit Dennisisspiderman’s underlying hypothesis?
We assume that he somehow thinks that the game compensates for a large rating gap with increased aggressiveness. But if that’s the case, we would expect to see increasing aggressiveness as the rating gap grows. And that’s definitely not the case.
And therefore, the observations really doesn’t fit Dennisisspiderman’s hypothesis particularly well. On the other hand, they seem to fit together like hand in glove with the hypothesis that the changes in aggressiveness are random.
Answering the right question with the wrong data
Let’s assume that Dennisisspiderman had a sufficient dataset and was able to prove that the AI’s level og aggression increases with the rating gap. What would that really tell us?
We don’t see any facts suggesting that chasing the ball more aggressively right in fact gives the AI-team an advantage. Increased aggression usually comes with a risk of running out of energy and thereby leave yourself vulnerable to counter attacks. So, even if we were to establish that the AI acted more aggressively against superior opponents, there is no justification for the claim that this is likely to work as a handicap.
In addition to that, it is very obvious that this conclusion can’t be extrapolated to a multiplayer setting where human player, not the AI, is deciding whether the players should chase the ball.
Hence, we really don’t see that this experiment, even if it had a sufficient size, would tell us anything with relevancy to the handicapping debate or for that matter bronze benching.
We find that Dennisisspiderman’s experiment has limited bearing on the likelihood of the handicapping claim as well as the bronze benching claim. However, we find that there is an obvious need for experiments which contain systematic studies of the AI behavior. Therefore, we can only encourage people to make more studies like these.