Google's artificial intelligence division announced the creation of a new method for training neural networks, combining the use of advanced algorithms and old video games. Old Atari video games are used as learning environment.
The developers of DeepMind (recall that these people created the AlphaGo neural network, which has repeatedly defeated the best players in the logic game of go) believe that machines can learn in the same way as humans. Using the DMLab-30 training system, based on the Quake III shooter and Atari arcade games (57 different games are used), engineers have developed a new IMPALA (Importance Weighted Actor-Learner Architectures) machine learning algorithm. It allows individual parts to learn how to perform several tasks at once, and then exchange knowledge among themselves.
In many ways, the new system was based on the earlier Asynchronous Actor-Critic Agents (A3C) architecture system, in which individual agents explore the environment, then the process is paused and they exchange knowledge with the central component, the "student". As for IMPALA, it may have more agents, and the learning process itself takes place in a slightly different way. In it, agents send information to two "students" at once, who then also exchange data with each other. In addition, if in A3C the calculation of the gradient of the loss function (in other words, the discrepancy between the predicted and obtained parameter values) is done by the agents themselves, who send information to the central core, then in the IMPALA system, this task is done by "students".
An example of a person playing through the game:
Here's how the IMPALA system handles the same task:
One of the main challenges in developing AI is time and the need for high computing power. Even when autonomous, machines need rules that they can follow in their own experiments and finding ways to solve problems. Since we can't just build robots and let them learn, developers use simulations and deep learning techniques.
In order for modern neural networks to learn something, they have to process a huge amount of information, in this case, billions of frames. And the faster they do it, the less time it takes to learn.
Promotional video:
With enough processors, DeepMind says IMPALA achieves 250,000 frames per second, or 21 billion frames per day. This is an absolute record for tasks of this kind, according to The Next Web. The developers themselves comment that their AI system copes with the task better than similar machines and people.
In the future, similar AI algorithms can be used in robotics. By optimizing machine learning systems, robots will adapt to their environment faster and work more efficiently.
Nikolay Khizhnyak
