Many of us remember and know what kindergartens are like. The learning rooms are crammed with toys and puzzles, music and books, flowers and even sometimes cats, thus presenting a rich and lush world with which children can play and learn through play. But contrary to popular belief, children's play is far from easy. They don't just have fun - they learn, they gain an understanding of the world. By playing in a diverse and delightful universe, we nurture a child's multifaceted intelligence.
Why not teach AI in the same way?
Not long ago, the non-profit institute OpenAI opened up the virtual world for AI to explore and play with. The goal of the Universe project is as big as its name: to teach a single AI to perform any task that a person can perform using a computer.
By training individual AI agents to excel at various real-world tasks, OpenAI hopes to bring us one step closer to truly intelligent bots - with flexible reasoning skills like we do.
General level intelligence
There is no doubt that AI is becoming frighteningly smart.
Promotional video:
Computers can now accurately see, hear and translate languages, sometimes even ahead of humans. Just earlier this year, in a series of high-profile go games, DeepMind's AlphaGo defeated 18-time world champion Lee Sedol, ten years earlier than some experts expected.
But the truth is, AIs are only as good as taught. Ask AlphaGo to play chess, and the program will most likely get confused in a machine-like manner, even if you explain the rules to it in great detail.
So far, our AI systems are super-efficient horses for one trick. The teaching method is partly to blame: Researchers start with a blank slate of AI, guiding it through millions of trials, until it succeeds and solves one problem. AI never experiences something else, so how does it know how to solve any other problem?
To get to the intelligence of the general level - capable of using the experience gained on the human level to solve new problems - AI needs to transfer its experience to solving other problems. And the Universe will help them with this. By experiencing a world full of different scenarios, OpenAI scientists hope that AI will gain knowledge of the world and flexible problem-solving skills that allow it to “think” rather than being stuck in a single loop forever.
Brave new world
At its core, Universe is a powerful platform that includes thousands of environments, typically providing standard methods for training AI agents. As a software platform, Universe provides a platform for running someone else's software so that programs can learn in different environments - Atari and flash games, applications and websites, for example, are already adopted.
There will be others ahead.
In theory, the Universe can run any software on any computer, allowing scientists to insert and train their AIs at will. It's like sending a child to a summer camp: you choose your niche, type of activity, wait for him to master it, then another, and so on, take a dip and repeat.
In the Universe, AI interacts with the virtual world in the same way that humans use a computer: it “sees” pixels on the screen and uses a virtual keyboard and mouse to enter commands.
This is made possible by Virtual Network Computing (VNC), essentially a desktop sharing system that allows keyboard and mouse movements to be transmitted from one computer (AI) to another (learning environment). When the environment changes, VNC sends updated screenshots back to the AI, allowing it to take the next step. VNC acts as the eyes and hands of the AI.
How does the training take place?
All AIs that are connected to the Universe are trained using what is known as reinforcement learning, a powerful technique that has led to the success of AlphaGo. However, this term hides the way we humans train dolphins, dogs and even children. This is learning by trial and error: choose an action, and if you've been rewarded for it, keep up the good work. If not, try something else.
Instead of starting with a completely blank AI, researchers sometimes give them a boost by letting them “watch” people solve a problem. This allows the AI to form a first impression and have a better understanding of how to optimize its decisions.
Reinforcement learning is already being used in many AI applications. Inside the Universe, however, the power of this technology is revealed to its fullest. Because AI can jump between games and applications, it can take what it learned in one application and easily use it to figure it out in another - called transfer learning or transfer learning. This skill is not easy to master, but it is essential on the road to intelligent cars.
We're slowly moving there, according to OpenAI: some of their agents are already showing signs of shifting learning from one driving game to another.
From games to the world of beats
Like many other AI developers, OpenAI uses games to propel the Universe for a reason: it's simply measured in terms of success. Since games are measured by various statistics and scores, the system can easily use these numbers to gauge the progress of the AI and reward it accordingly. This is essential for reinforcement learning.
Since the Universe relies on pixels and keyboards, humans can play games on the platform too. These sessions are recorded and provide a baseline for assessing AI performance (not a bad job, you must admit)
But games are just a small part of our interaction with the digital world, and the Universe is already moving beyond its limits with the Mini World of Bits project. Bits are a collection of different browser interactions that we encounter as we surf the web: when we type text or select options from drop-down menus by clicking submit.
These tasks, while simple, form the foundation of how we connect to the treasure trove of the Web. OpenAI wants AI to move freely on the Internet - for example, it can order a plane ticket. In one of the Universe environments, researchers are already giving the AI the desired booking schedule and teaching it to search for flights on various airlines.
And this is just the beginning.
The Universe is only growing and expanding. Microsoft's Malmo platform, which uses Minecraft to train AI, needs to integrate with the Universe. The popular protein folding game fold.it, Android apps, HTML5 games and more are waiting in line.
Ghost in the car
So now we can teach AI to play different games and surf the web. Great thing, just think. Will this lead us to general intelligence?
Perhaps the road will be long.
But an AI that knows how to win any game you throw at it can only think logically and achieve victory in a few steps. An AI that can navigate the chaotic world of GTA V should already understand the basics of real-world physics, violence, and retaliation. An AI that can run on the Internet already knows how people usually communicate with each other and can use that knowledge to gain information, create its own web identity, or even look into yours.
Every day we learn, play, work and grow in the digital realm. For many, the world of ones and zeros is as real as the one we were born into. Now that AI has access to this digital world, it's its turn to grow. Let's see how far he can go.
ILYA KHEL