Computer scientist Christoph Solge is trying to eliminate the need for rules that govern the behavior of robots. His strategy is to give them a goal: to make us more powerful. Christophe works at the Game Innovation Lab at New York University. Sasha Maslov interviewed Sold for Quanta Magazine, from which we learn that we may not be worth trying to hold back the unstoppable stream of technological singularity.
Isaac Asimov's famous Three Laws of Robotics - limiting the behavior of androids and automata to keep humanity safe - were also unfinished. These laws first appeared in Asimov's story in 1942, and then in classical works like "I, Robot" and sound something like this:
1. A robot cannot harm a person or, by its inaction, allow harm to be done to a person.
2. The robot must obey orders given by humans, if such do not contradict the First Law.
3. A robot must defend its existence, if this protection does not contradict the First or Second Law.
Of course, in these laws one can find many contradictions and loopholes (which, in fact, Azimov himself used). In our current age of advanced software with machine learning and autonomous robotics, defining and implementing the iron ethics of artificial intelligence has become a pressing challenge for organizations such as the Machine Intelligence Research Institute and OpenAI.
Christoph Salge took a different approach. Instead of imposing top-down philosophical definitions of how artificial agents should or should not behave, Salge and his colleague Daniel Polani are exploring the bottom-up path, or “what the robot should do first,” as they wrote in his article "Empowerment as a Substitute for the Three Laws of Robotics." Empowerment, a concept born at the intersection of cybernetics and psychology, describes the intrinsic motivation of an agent to resist and work in the environment at the same time. “As an organism, it wants to survive. He wants to leave a mark on the world,”explains Salge. Roomba's vacuum cleaner, programmed to search for a charging station when its batteries are low, is a rudimentary example of "empowered": to continue to function in the world,he must receive a charge and continue his own existence, that is, survive.
Empowerment might sound like a recipe for the very outcome that safe AI advocates like Nick Bostrom fear: a powerful autonomous system, concerned only with self-interest, and going crazy in the process. But Salge, who studies human-machine social interactions, asks: What happens if an empowered agent “also empowers another? The robot needs to not only want to stay in working order - it needs to also want to support its human partner."
Promotional video:
Solge and Polanyi realized that information theory offered a way to translate this mutual expansion into a mathematical framework for a non-philosophizing artificial agent. “One of the downsides of the three laws of robotics is that they are language-based, and the language is highly ambiguous,” says Salge. "We're trying to find something that can actually be done."
Some technologists believe that AI is a major, even catastrophic, threat to human existence. And you?
I will refrain. So I really believe that there is now a fear of robots and the growing influence of AI. But I think in the short term we are likely to be more concerned about possible job changes, the decision-making process, the loss of democracy, the loss of privacy. I don't know how likely an unstoppable AI will emerge anytime soon. But even if AI is to control the healthcare system and issue prescriptions, we need to think about the ethical issues that arise in the course of its operation.
How can the empowerment concept help us deal with these challenges?
I think the idea of empowerment fills a niche. She will not let the agent let the person die, but once you can hold on to that threshold, she will support the intention to create additional opportunities for the person to express and influence the world. In one of Asimov's books, robots will simply end up putting all humans in safe containers. This would be undesirable. If our ability to influence the world continues to improve, I think it will be a much more interesting goal to achieve.
You tested your ideas of virtual agents in a video game environment. What happened?
The agent, motivated by his own extended rights, will dodge the projectile and will not fall into the pit, in general, will avoid any situation that could lead to his loss of mobility, death or damage in such a way as to reduce his operability. He'll just hold on.
In conjunction with a human player, who is also endowed with enhanced rights, we saw that the virtual robot would maintain a certain distance so as not to impede human movement. He will not block you, he will not stand in the aisle so that you cannot pass. He will stay as close to you as possible to help you. This leads to behavior in which he can both take the initiative and follow someone else's.
For example, we created a scenario in which we had a laser barrier that is dangerous for humans, but safe for a robot. If the person in this game approaches the lasers, the robot has more incentive to block the laser. The stimulus is enhanced when a person stands directly in front of the barrier, as if intending to cross it. And the robot actually blocks the laser, standing in front of the person.
Did these agents exhibit any unintentional behavior similar to what follows from the three laws in Asimov's book?
The behavior was good at first. For example, a virtual robot intercepted opponents who tried to kill you. From time to time he would jump under a bullet if that was the only way to save you. But what surprised us especially from the very beginning was that he was also very afraid of the person.
The reason for this is due to his "short-sighted" model: in fact, he analyzes how sequences of certain actions in two or three steps can affect the world, both for you and him. Therefore, in the first step, we programmed that the player acts randomly. But in practice, this resulted in the fact that the agent treated the person as a kind of psychopath, who could, for example, shoot the agent at any time. Therefore, the agent had to very, very carefully choose the position in which the person could not kill him.
We needed to fix this, so we modeled the so-called trust assumption. Basically, the companion agent acts on the assumption that the person will only choose those actions that will not limit the extended rights of the agent himself - perhaps this is a more suitable model for the companion.
We also noticed that if you had, say, 10 health points in the game, the companion didn't really care if you lost eight or nine of those points - he could even shoot you once, just for fun. And then we realized that there is a gap between the world in which we live and the model in a computer game. As soon as we modeled the capacity limitation caused by the loss of health, the problem went away. It could also be solved by creating a model that was not so short-sighted that could calculate actions a couple more steps into the future. If the agent could look further into the future, he would see that having more health points could be beneficial for future events.
Taking into account that changing the number of health points does not affect my extended rights in any way, the agent decides: "I shoot him, I don't shoot - what's the difference?" And sometimes it shoots. Which, of course, is a problem. I don't want random shots at players. We've added a fix to make the virtual robot a little more worried about your condition than its own.
How do you make these concepts accurate?
If we consider agents as control systems, they can be decomposed into information components: in the world, something is happening and in one way or another concerns you. We are talking about information not as things that you perceive, but as influences of any kind - it can be a substance, something flowing between the world and you. There may be temperature or nutrients in your body. Any thing that crosses the border between the world and the agent carries information in itself. In the same way, an agent can influence the outside world in a variety of ways, also outputting information into it.
You can consider this stream as the bandwidth of the channel, this is a concept from information theory. You can have wide powers, extended rights, if you are able to take different actions that lead to different results. If something goes wrong, you will lose your authority, because the loss of capacity corresponds to a quantitative decrease in the bandwidth between you and the environment. This is the main idea.
How much does an agent need to know for his extended powers to take full effect?
Extended rights have the advantage that they can be applied even when you do not have full knowledge. The agent really needs a model of how his actions will affect the world, but he does not need a complete understanding of the world and all its subtleties. Unlike some approaches that try to model everything in the world as much as possible, in our case you just need to find out how your actions affect your own perception. You don't need to learn everything about everything; you only need an agent who explores the world. He does something and tries to understand how his actions affect the world. The model grows, and the agent understands better and better where the boundaries of his authority extend.
You tested it in a virtual environment. Why not in the real world?
The main obstacle to scaling this model and placing it in a real robot is the difficulty of calculating the bandwidth of an agent and a person in such a rich environment as the real world for a long time to come. All these processes have yet to become effective. I am optimistic, but so far this problem remains purely computational. Therefore, we check the operation of the system in a computer game, in a simplified form.
It looks like empowerment, ideally, will make our machines powerful service dogs
I even know some robot technicians who deliberately model the behavior of a dog-based companion. I think if robots treat us like our dogs, in this future we can all get along.
Ilya Khel