Roco's Basilisk is a god-like form of artificial intelligence, so dangerous that if you just think about it, you will spend the rest of your days in horrific torture. It looks like a cassette from the movie "The Ring". However, even death will not be a deliverance, because the Basilisk Roko will resurrect you and continue his torture. T&P presents a translation of an article about one of the strangest legends generated on the Internet.
WARNING: After reading this article, you can condemn yourself to eternal suffering and torment.
Roco's Basilisk appeared at the junction of a philosophical thought experiment and urban legend. The first mention of him appeared on the Less Wrong discussion board, which brings together people interested in optimizing thinking and life through the prism of mathematics and rationality. Its creators are significant figures in techno-futurism, and among the achievements of their research institute is contributing to the academic debate about technological ethics and decision theory. However, what you are about to read now may seem strange and even crazy. Despite this, very influential and wealthy scientists believe in this.
One day, a user named Roko posted the following thought experiment: What if, in the future, a malicious artificial intelligence emerges that wants to punish those who don't follow its orders? And what if he wants to punish those people who in the past did not contribute to his creation? Would Less Wrong readers then help the evil AI come into being, or would they be doomed to eternal torment?
Less Wrong founder Eliezer Yudkowski took Roko's statement with anger. This is what he replied to him: “You must be smart if you came to this idea. However, it saddens me that people who are smart enough to imagine such a thing are not smart enough to KEEP THEIR STUPID TONGUE BY TEETH and not tell anyone about it, because this is more important than showing yourself smart, telling all your friends about it …
"What if in the future there is an artificial intelligence that wants to punish those who do not follow its orders?"
Yudkowski admitted that Rocko was guilty of the nightmares of Less Wrong users who had read the thread and deleted it, making Rocko's Basilisk legendary. This thought experiment became so dangerous that thinking about it threatened the user's mental health.
What is Less Wrong doing? Formation of the concept of the future of humanity based on the singularity. They believe that computing power in the future will become so high that artificial intelligence can be created with the help of a computer - and with it the ability to download human consciousness to a hard drive. The term "singularity" originated in 1958 during a discussion between two geniuses of mathematics - Stanislav Ulam and John von Neumann, when Neumann said: "The constantly accelerating progress of technology will make it possible to approach a singularity in which technology cannot be understood by humans." Futurists and science fiction writers like Vernor Vinge and Raymond Kurzweil popularized the term because they believed that the Singularity was waiting for us all very soon - in the next 50 years. While Kurzweil prepares for the singularityYudkowski has high hopes for cryonics: "If you didn't register your children for cryopreservation in advance, you are lousy parents."
Promotional video:
If you believe that a singularity is coming and that powerful AIs will appear in the near future, a natural question arises: will they be friendly or evil? The Yudkowski Foundation aims to direct the development of technology in such a way that we have exactly friendly artificial intelligence. This question is of paramount importance to him and to many others. The Singularity will lead us to a god-equivalent machine.
However, this does not explain why Roco's Basilisk looks so terrible in the eyes of these people. The question requires a look at Less Wrong's main "dogma" - "Timeless Decision Theory" (HTDM). VPPR is a guide to rational action, based on game theory, Bayesian probability, and decision theory, but with parallel universes and quantum mechanics in mind. VTPR grew out of a classic thought experiment, the Newcomb Paradox, in which an alien with superintelligence offers you two boxes. He gives you a choice: either take both boxes, or just box B. If you choose both, then you are guaranteed to receive a thousand dollars. If you only take box B, you may not get anything. However, the alien has one more trick in store: he has an all-knowing supercomputer that made a prediction a week ago,whether you take both boxes or just B. If the computer predicted that you would take both boxes, the alien would leave the other empty. If the computer predicted that you would choose Box B, it would put one million dollars in there.
So what are you going to do? Remember that the supercomputer is omniscient.
This problem has confused many theorists. The alien can no longer change the contents of the boxes. The safest way is to take both boxes and get your thousand. But what if the computer is really omniscient? Then all you have to do is take box B to get a million. But if he turns out to be wrong? And no matter what the computer predicted, is there really no way to change your destiny? Then, damn it, you have to take both boxes. But in that case …
The maddening paradox that forces us to choose between free will and divine prediction has no resolution, and people can only shrug their shoulders and choose the option that is most comfortable for them. WTPR gives advice: take box B. Even if the alien decided to laugh at you and open an empty box with the words: "The computer predicted that you will take both boxes, ha ha!" - you still have to choose it. The rationale for this is as follows: in order to make a prediction, the computer had to simulate the entire universe, including you. Thus, at this moment, standing in front of the boxes, you can be just a simulation of the computer, but what you do will affect reality (or reality). So take Box B and get a million.
"The maddening paradox that forces us to choose between free will and divine prediction has no resolution."
What does this have to do with the Basilisk Roco? Well, he has a couple of boxes for you too. Maybe you are right now in a simulation created by Basilisk. Then perhaps we get a slightly modified version of Newcome's paradox: Roco's Basilisk tells you that if you take Box B, you will suffer eternal torment. If you take both boxes, then you will be forced to devote your life to the creation of the Basilisk. If the Basilisk actually exists (or, worse, it already exists and is the god of this reality), he will see that you have not chosen the option of helping in its creation and will punish you.
You might be wondering why this question is so important to Less Wrong, given the convention of this thought experiment. It's not a fact that Rocko's Basilisk will ever be created. However, Yudkowski removed references to Roko's Basilisk not because he believes in its existence or an imminent invention, but because he considers the idea of a Basilisk dangerous to humanity.
As such, Rocko's Basilisk is only dangerous to those who believe in it - as a result, the members of Less Wrong who support the Basilisk idea have a kind of forbidden knowledge, which reminds us of Lovecraft's scary stories about Cthulhu or the Necronomicon. However, unless you subscribe to all of these theories and feel tempted to obey an insidious machine from the future, Roco's Basilisk poses no threat to you.
I am more concerned about people who think they have risen above conventional moral standards. Like Yudkowski's expected friendly AI, he is himself a utilitarian: he believes that the greatest good for all of humanity is ethically justified, even if several people must die or suffer on the way to it. Not everyone can face such a choice, but the point is different: what if a journalist writes about a thought experiment that can destroy the consciousness of people, thereby harming humanity and hindering progress in the development of artificial intelligence and the singularity? In this case, any good that I have done in my life must outweigh the harm that I have done to the world. Or, perhaps, Yudkowski of the future, who rose from cryogenic sleep, merged with the singularity and decided to simulate me in the following situation:the first box - I will write this article, the second - no. Please, almighty Yudkowski, don't hurt me.