You could not have noticed this, but in the first half of August 2017, a small coup happened over the weekend. In the evening on Friday, in front of a crowd of many thousands, AI-bot won a professional player in Dota 2 – one of the most popular video games in the world. The human champion, polite guy Danil "Dandy" Ishutin surrendered after he was killed three times, and said that he could not defeat the uncontrollable bot. "He's a bit like a man," the Dandy said. "But at the same time it's similar to something else."
The bot's father was none other than techno-industrialist Ilon Mask, who helped finance and found the organization that developed it, OpenAI. He was not at the event, but he expressed his attitude on Twitter.
OpenAI was the first to win the best players in the world in competitive e-sports. This is much more complex than traditional board games like chess or go.
What's more interesting, OpenAI has learned everything that he knows. He trained, constantly playing with himself, accumulating numerous "careers" in the game experience in just two weeks.
What follows from this? Was the Friday show more impressive than the victory of Google AI in the board game go? If briefly, then probably not, but still it represents a significant step forward – both for e-sports and for the world of AI.
Yes, video games More difficult is chess
First, you need to consider the statement of the Mask that Dota "is much more complex than traditional board games like chess or go." This is really so. Real-time battles and strategic games such as Dota and Starcraft II are complex challenges that computers can not yet cope with. These games require strategic thinking, and, unlike desktops, conceal important information from players. You see everything that happens on the chessboard, but not in the video game. This means that you need to predict and anticipate everything that your opponent can do. This requires imagination and intuition.
In Dota, this complexity increases with the work of people in teams of five people, coordinating strategies that change in the course of action depending on the characters used. To further complicate the task, the game has more than 100 different characters, each of which has a unique set of skills; Characters can be equipped with different unique items, each of which, when applied at the right time, can lead to a win. All this means that it is impossible to program winning strategies in a bot for Dota.
But the game that OpenAI played was not that complicated. Instead of "5 by 5" he played with people in "1 on 1"; Instead of choosing a character, the person and computer had the same hero – a friend named Shadow Fiend [Демон тени]a set of attacks which is fairly straightforward. My colleague Vlad Savov, who sat down on Dota, also described his impression of the Friday game, said that the match "1 on 1" represents "only a small part of the complexity of the full competition." So it's probably not as hard as go.
You can not think of a better calculator
The second big catch is the advantages that OpenAI has over a person. One of the main disputes in the AI community was the discussion of whether the bot had access to the API for Dota bots – this would allow it to connect directly to the flow of information from the game, to parameters such as, for example, the distance between the players. Greg Brockman of OpenAI confirmed to our publication that the AI actually used the API, and that certain techniques were stiffly sewn into the agent, including the items he used in the game. He was also taught some strategies, using the trial and error technique called "stimulated learning". In general, he was trained a little.
Andreas Teodorou, AI researcher in games from Bath University and an experienced player in Dota, explains why it matters. "One of the main features of Dota – you need to calculate distances to know how far some attacks spread," he says. The API allows bots to estimate distances. So you can say: 'If someone is 500 meters away, do it', but the person has to calculate everything himself, learning by trial and error. If the bots have access to information that no one has, this gives them an advantage. " This is especially important for a 1-on-1 game with a character such as Shadow Fiend, where players have to concentrate on choosing the right time for attacks, rather than on a common strategy.
Brockman says that learning this skill for AI – the task is trivial, and it has never been the main one for research in OpenAI. He says that the institute bot would have coped without information from the API, but "he would have simply spent much more time acquiring vision skills, which is already working, so what's the point?"
Some skills can be Learn, but can not be taught
With all this in mind, is it possible to defeat the victory of OpenAI? No way, says Brockman. He points out that the way he studied independently was more important than the victory itself. Previous champions of AI type AlphaGo learned to play games, processing past matches of champions, and the bot of OpenAI itself learned (almost) everything that it knows.
"You have a system that simply played against yourself, and developed enough reliable strategies To defeat professionals. This can not be taken for granted, says Brockman. – And this is a big question for any machine learning system: how does complexity get into the model? Where does it come from? "
According to him, the OpenAI bot shows that we do not need to train computers with complex things: they can do it themselves. And although some behavior of the bot was pre-programmed, he developed some strategies himself. For example, he learned how to cheat opponents, pretending to start an attack, but canceling it at the last moment, and forcing a person to repulse an attack that did not happen – just like a feint in the box.
Others estimate this more skeptically. AI researcher Denny Britz, who wrote a popular blog post on this topic, told us that it's difficult to assess the degree of achievement without knowing the technical details. Brockman says that they will follow, but when exactly, he could not say. "Before the release of the work, it is not clear what the achievements were from a technical point of view," says Britz.
Teodorou points out that although the OpenAI bot won the Dandy in the competition, when the players looked at his tactics, they were able to outwit him. "If you study their strategies, it's clear that they played, not like everyone else, and won," he says. Players used non-standard strategies – they would not surprise a person, but AI has not seen them yet. "The boat was not flexible enough," says Teodorou. Brockman argues that after studying the new strategies, the bot would not succumb to them again.
All experts agree that this was a serious achievement, but the real difficulties are just beginning. This will be a "5 by 5" match, where the agents of OpenAI will not only have to deal with duels in the middle of the map, but also work on a sprawling, chaotic battlefield with many heroes, dozens of support units and unexpected turns. Brockman says that OpenAI is now targeting the Dota tournament next year, which will take place in 12 months. And during this time, you need to spend much more training.