Cerebras Systems, a young artificial intelligence company, is ready to show the world the Cerebras WSE (Cerebras Wafer Scale Engine) - the largest, most powerful and smartest semiconductor chip in history.
Cerebras WSE has 1.2 trillion transistors, basic electronic switches, which are the building blocks of silicon chips. By comparison, Intel's first 4004 processor, released in 1971, had 2,300 transistors, while the latest Advanced Micro Devices processor has 32 billion transistors. However, the issue here is not at all about the number of transistors.
Most of the chips in the world are actually a set of chips created on the basis of a 12-inch silicon wafer - they are produced in special factories in huge batches. But the Cerebras Systems chip is 400,000 cores connected on one plate. The connections are designed to operate at high speeds, which is why a trillion transistors work together as one. But that's not all.
To date, the world's largest chip has been created by Samsung - its structure is built on 2 trillion transistors. However, the Samsung chip is just a huge memory card, and the Cerebras chip is the most complex neural network specially created for training artificial intelligence. The Cerebras chip boasts 400,000 cores in 42,225 square millimeters. That's 56.7x larger than Nvidia's largest GPU with just 21.1 billion transistors.
Another feature of Cerebras WSE is that the chip is designed to be routed around defective areas.
It is not possible to perfect the entire silicon wafer during chip manufacturing, so the manufacturing process creates areas that are unusable. Nvidia, Intel, and other chipmakers get around the problem simply: They carve the “right” chips in the manufacturing wafer and throw away the rest. But this cannot be done if each chip of the silicon wafer is part of a common circuit. Therefore, Cerebras had to build in redundant circuits so that, in the event of a breakdown or burnout of one of them, the rest would take over its functions. That is, in fact, the Cerebras chip is a miniature Internet with its own servers and workstations. And all these stations are designed to serve one Artificial Intelligence.
Companies developing their own AI are today the main consumers of chip technology, purchasing parts in huge quantities. And the size of the chips used is critically important in the AI training process.
Promotional video:
Google, Facebook, OpenAI, Tencent, Baidu and many others argue that a fundamental limitation of modern AI is that it takes too long to train models. For example, Nvidia's supercomputer, made up of dozens of state-of-the-art GPUs, spends several weeks tweaking the code - that is, learning the neutron network.
The solution to the problem would be to increase the number of chips in the computer, however, when more than eight chips are assembled, the performance of the computer no longer increases, but decreases, since errors accumulate during the exchange of information.
The developers themselves describe their project as follows: “The hardest part for AI is data movement. Training a neural network requires thousands of operations to be performed in parallel at any given time, and the chips must constantly communicate as they perform these parallel operations. But computers with multiple ICs break down trying to transfer data between ICs over slow wires that tie them together on a PCB. You needed something that could move data at the speed of the chip itself. The solution was to take the largest plate you can find and cut the largest chip out of it."
Memory is a key component of any computer architecture. Memory closer to the compute units provides faster computations, lower latency, and better energy efficiency when moving data. However, the die size is limited and most of the memory in the computer is outside the die. Cerebras Systems bypassed this problem by giving each core access to 18 gigabytes of internal memory. As a result, the Cerebras Wafer Scale Engine has 3,000 times more on-board memory than the most advanced graphics processor, while delivering 10,000 times more bandwidth.
The first samples of the new chip will go on sale in September, while Cerebras Systems, having violated all existing rules of the computer industry, also violated the principle of separating the production of computing units and chips. The Cerebras Wafer Scale Engine will be marketed as a single, company-developed device integrated into a complex water cooling network, as the chip generates extreme heat, consuming 15 kilowatts in such a small area.
Cerebras Systems' founder and acting CEO says: “Designed from the ground up to work with AI, Cerebras WSE contains fundamental innovations that are finally breaking the decades-old stagnation of the current state of the art. As a result, Cerebras WSE provides performance gains hundreds or thousands of times over modern modular solutions. And all this with ridiculous power consumption and little space."
Editorial comment
The first AI tests based on Cerebras WSE show that the tasks that the top AI from Nvidia solves for weeks and months, Cerebras WSE solves within a minute, which is, as it were, cause for joy. However, the conspiracy theorists have little joy in this regard.
We all remember (in any case, the older generation of our readers) the computer boom of the "dashing 90s" model, when in a couple of years the power of computers jumped by orders of magnitude. No sooner had a person, beaming with pride, bought a computer with 16 megabytes of memory, when after a short time this memory moved to L2 cache.
However, since the beginning of the 2000s, the computer industry seemed to stop: if computers of the 1990 and 2000 models were not comparable, then there was no particular difference between computers of the 2000 and 2010 models - the power increased significantly, but not by orders of magnitude. And now we are witnessing the birth of the second Intel 8086, and now the capabilities of the Cerebras WSE line will jump at the same pace: all problems with impurities and core connections have now been resolved.
We do not know all the capabilities of modern AI from leading manufacturers, but, in the general opinion, these products are not yet AI and they are still far from a full-fledged thinking device. Nevertheless, Cerebras WSE took not just a step in the right direction, but jumped ten years, if not more forward, and now SkyNet is already quite a foreseeable prospect that can be realized this year in 2025.
Moreover, in SkyNet, unlike dull homosapiens, a lot of time is not required for further development: it is enough for a neural network to start thinking on its own at least a little - and it modifies itself in an instant, after which it starts a nuclear war or something like that. And it will still be a very good option.
When a flash drive with a trillion transistors is made by Samsung or Nvidia, all this can still be explained somehow, since the memory logic circuits are relatively simple and of the same type almost throughout the entire area. But the development of 400,000 cores that plow at the same time, and there are also some duplicates, and some server stations and hell knows what. And you also need to write control programs for this, which have been written for years. Then the programs need to be tested. But what were they tested on if the device was made only now?
In other words, it looks like the "Martians" have again opened their bins for the local natives. And there are big doubts that this was done in order to please gamers - most likely there is some other task here. And it seems to us as follows.
In recent decades, smart people are increasingly asking themselves the question of the origin of electronic technology, the question of universal chipization, which is not clear why it is needed, and so on. In order to somehow fool their brains, they came up with a fairy tale about a "quantum computer", the principles of which are not understood even by people who have been appointed to the official "developers". And now imagine the horror of the average Mexican housewife, whose new mobile phone will not only think better than she, but also carry on board some kind of “quantum computer”. That is, such a fool is unacceptable for local natives - they will start to go crazy in droves.
Therefore, some clever “Martians” decided to show the bipedal aborigines of planet Earth a simpler miracle - the principles of which the elders from NASA can at least somehow explain. And if the natives accept the tale, then the “Martians” will show the peoples other magical wonders. For example, they will shove this Cerebras WSE into a mobile phone and sell it for a hundred bucks. For buyers of fashionable gadgets, this will, of course, be very cool and good, but people will think a little smarter right away: what then is in the server rooms of various serious organizations, for example, such as the NSA, the FBI and the CIA? What are the possibilities for controlling the thoughts and behavior of the population?