PySC2 editor shows the interpretation of the game field for the person (left), as well as the color versions of the layers of signs on the right. For example, the top row shows signs of the height of the surface, the "fog of war", slime, camera location and player resources, video
Testing AI agents in computer games is an important part of AI training. DeepMind pays great attention to this aspect of learning, using both specially created environments like DeepMind Lab, and well-known games like Atari and go (AlphaGo system). It is especially important to test agents in such games that were not specifically created for machine learning, but rather they were created for games people and people play well in them. Here, the training of AI is most valuable.
Based on these prerequisites, DeepMind together with Blizzard Entertainment released a set of SC2LE tools to stimulate research in the field of AI on the StarCraft II platform.
StarCraft and StarCraft II are among the most popular games of all time, tournaments for them take more than 20 years. The original version of StarCraft is also used for machine learning and AI research, and developers show their creations at the annual AIIDE bots competition. Part of the success of the game is associated with good balance and multi-level gameplay, which makes it an ideal option for AI research.
So, the main task of the player is to defeat the opponent, but at the same time you need to perform a lot of subtasks: to extract resources, build buildings. The game has other qualities, attractive for the development of AI: for example, a constant large pool of experienced online players, where you can hone your skills.
The SC2LE toolkit includes The following:
- Programming interfaces of the Machine Learning API developed by Blizzard, through which developers and researchers can connect to the game engine. Including the first released tools for Linux.
- A set of data for training with anonymous games records. DeepMind promises in the coming weeks to increase the number of records from 65 thousand to 500 thousand
- Open source version of the DeepMind toolkit – PySC2, to easily use the API and layer layers of features with other agents.
- Sets of simple mini-games for testing the performance of agents on simple tasks.
- A joint scientific paper with the description of the environment and the basic results of machine learning in mini-games, the description of training with the teacher on the set of data of games records and a full-fledged game of 1v1 agent against game AI
StarCraft II will be a difficult game for learning AI. Suffice it to say that more than 300 basic actions are available for the player. Compare this with Atari games, where the number of actions does not exceed ten (such as "up", "down", "left", "right"). In addition, actions in StarCraft have a hierarchical structure, can be changed or supplemented, and many of them require pointing on the screen. Simple mathematics shows that even on a fragment of 84 × 84 pixels there are about 100 million possible actions!
"StarCraft is interesting for many reasons," says Oriol Vinyals, lead researcher at DeepMind On the project StarCraft II and the expert-player in StarCraft II (powerful bots for the game he wrote back in his student years). – Memory is a critical factor. What you see at the moment is different from what you saw before, and something specific that happened a moment ago can make you change your behavior at the moment. "
The PySC2 editor provides an easy-to-use Interface for connecting agents to the game. As shown in the very first screenshot, the game has been decomposed into "feature layers" that are isolated from each other. These are attributes such as unit types, visibility on the map, surface height, etc.
The animation below shows some of the mini-games designed to train agents for specific actions in the game, such as moving the camera, collecting minerals or Selection of units. Developers from DeepMind hope that the community will throw ideas for new mini-games.
The first results show that AI agents do a good job with mini-games, but in the whole game even The best agents like the A3C can not win against the built-in AI even at the simplest level. Perhaps gathering more gaming sessions and additional training of agents will help fix the situation. At the same time, training on such a large base (500,000 game sessions) will open up fundamentally new research opportunities, such as long-term AI memory and event sequence predictions.
Blizzard developers say they are interested in opening a game engine for External AI agents. First, it can make the game more interesting for current players. Secondly, it is important for studying the game process itself and developing future games.