Almost everyone who dives even a little into the topic of artificial intelligence comes to the conclusion that either it will lead us to a fiery apocalypse, or to a magical utopia. There are practically no options between. Of course, this is partly due to the fact that slogans like "The end is near!" or "Utopia is coming!" But still…
Part of it boils down to how people feel about change, especially big. Millennialism has nothing to do with being a millennial, being born in the 90s, and remembering the Buffy the Vampire Slayer series. It is a style of thinking about the future that is associated with a deep-seated sense of destiny. Millennialism is "the expectation that the world will be destroyed and replaced by a perfect world and that a redeemer will come who will bring down evil and comfort the righteous."
Millennial beliefs, accordingly, closely link the ideas of destruction and creation. Among them are the ideas of huge, apocalyptic, seismic shifts that will destroy the fabric of the old world and build something completely new. A similar belief system exists in many of the major religions of the world, and even in the not-quite atheist and agnostic religions who believe in technology.
See, for example, how futurists await a technological singularity. According to Ray Kurzweil, the Singularity is the creation of paradise. Everyone will become immortal thanks to biotechnology that will heal our diseases; our brains can be uploaded to the cloud; inequality and suffering will disappear as phenomena. “Destruction of the world” is being replaced by Silicon Valley's favorite term: disruption, or radical industry change. And as with other millennial beliefs, your final view depends on whether you are waiting for the end of the world or the birth of a utopia.
There are many good reasons to be skeptical about this kind of thinking. Perhaps the most compelling of these is that the beliefs of millennials simply reflect people's attitudes toward change; just look at how many variations of these beliefs have grown up in the world.
These beliefs are present in aspects of Christian theology, although they became popular in their modern form in the 19th and 20th centuries. Ideas like Eternal Sorrow - many years of suffering and hardship - and the Rapture, when the righteous will be resurrected and the evil punished. After this destruction, the world will be rebuilt, or people will go to heaven.
Despite dogmatic atheism, there were many similar beliefs in Marxism. The only question is in relation to history. Just as believers are looking for signals that hint at the fulfillment of prophecy, Marxists are looking for signs that we are in the final stages of capitalism. They believe that society will inevitably degrade and degenerate to the very bottom - in fact, as Christians believe.
According to Marxism, when the exploitation of the working class by the rich becomes unsustainable, the working class gathers and overthrows the oppressor. "Sorrow" is replaced by "revolution". Sometimes revolutionary figures like Lenin or Marx himself are hailed as messiahs who bring the Millennium closer; their rhetoric inevitably contains calls for the destruction of the old system, on the ruins of which "we will build ours, we will build a new world." The righteous workers will get their rightful, and the wicked bourgeoisie will be destroyed.
Promotional video:
Even in Norse mythology there is an element of this, as James Hughes notes in his essay in Nick Bostrom's book, Global Catastrophic Risks. In Ragnarok, both humans and gods are defeated in the final apocalyptic battle, but since this is all a little gloomy, the Scandinavians added the idea of a new land on which the survivors will live in harmony.
Doomsday has also become a cultural trail. Take the ancient Egyptians and their beliefs about the afterlife; the lord of the underworld Osiris weighs the heart of a mortal together with a feather. If the heart of the deceased is too burdened by transgressions, the demon will eat it and the hope for an afterlife will disappear.
Perhaps something like this will happen during the singularity. As our technology improves, and therefore our strength, our hearts, the hearts of people, will be weighed against feathers. If they turn out to be too heavy - with stupidity, arrogance, prejudice, evil - we will fail the test and be destroyed. But if we go through and out of the singularity, heaven awaits us. As with other belief systems, there is no room for unbelievers; the whole society will change radically, whether you like it or not. Technological admiration.
It seems that every major development provokes such a response. And nuclear weapons too. Either it will be the last straw and we will destroy ourselves, or nuclear energy can be used to create a better world. In the early days of the nuclear age, people talked about electricity "too cheap to count." Scientists who worked on the bomb often thought that with such destructive power in the hands of man, we would simply have to get together and work together as a species.
When we see the same answer, over and over again, in different circumstances, arising in different fields, be it science, religion or politics, we need to consider human bias. We love the beliefs of millennials, so when the idea of artificial intelligence that surpasses human intelligence comes up, we immediately impose a familiar pattern.
We don't like facts. We don't like the information. We are not as rational as we think we are. We are creating narratives. Physicists observe the world, and we weave our own observations into narrative theories, stories about tiny billiard balls that fly here and there and collide with each other, or about space and time that bends, bends and expands. Historians try to give meaning to an endless stream of events. We love stories: stories laid out our past, our present, and they also prepare us for the future.
The millennial narrative is beautiful and compelling. He brings you social change. He can justify your daily suffering if you are grieving. He gives you hope that your life is important and meaningful. It gives you a sense of things progressing in a certain direction, according to the rules, and not just in chaos. He promises that the righteous will be saved and the heretics punished, even if there is suffering on the way. Finally, the millennial narrative promises heaven at the end of the tunnel.
We should be careful with the millennial narrative when we think about technological development, singularity, and existential risks. We shouted "wolves!" Many times when they were not there. Perhaps even now the world is not on the brink of disaster. Of course, this story is not so attractive. Of course, everyone wants an enchanting ending.
But dig deeper and you'll find that millennial beliefs aren't always the most promising, because they take the human agent out of the equation. We will have to believe in shades of gray and abandon the sinister apocalypse with the red-eyed AI and the fabulous utopia with the omnipotent AI that adores people.
Ilya Khel