Part One: The Human Colossus
Part Two: The Brain
Part Three: Flying Over the Nest of Neurons
Part four: neurocomputer interfaces
Part Five: The Neuaralink Problem
Part Six: Age of Wizards 1
Part Six: Age of Wizards 2
Part Seven: The Great Fusion
Promotional video:
Flying over the nest of neurons
This is Bock. Bock, thank you and your people for inventing language.
To thank you, we want to show you all the incredible things that we managed to build thanks to your invention.
Okay, let's put Bock on a plane, then into a submarine, then drag him to the top of the Burj Khalifa. Now let's show him a telescope, a TV and an iPhone. And let him sit on the Internet a little.
It was fun. How do you, Bock?
Yes, we understand that you are quite surprised. For dessert, let's show him how we communicate with each other.
Bock would be shocked if he found out that, despite all the magical abilities that people have acquired as a result of dialogues with each other, thanks to the ability to speak, the process of our communication is no different from what it was in his time. When two people are about to talk, they are using 50,000 year old technology.
Bock will also be surprised that in a world in which amazing machines work, the people who made these machines roam with the same biological bodies that Bock and his friends walked with. How is this possible?
This is why neurocomputer interfaces (BCIs) - a subset of the broader field of neural engineering that is itself a subset of biotechnology - are so interesting. We have repeatedly conquered the world with our technologies, but when it comes to brains - our main tool - the world of technology gives us nothing.
Therefore, we continue to communicate using the technology invented by Bock. Therefore, I am typing this sentence 20 times slower than I think, and therefore brain-related diseases still claim too many lives.
But 50,000 years after that great discovery, the world may change. The next frontier of the brain will be itself.
* * *
There are many different options for possible brain-computer interfaces (sometimes called brain-to-computer or brain-to-machine interfaces) that come in handy for different things. But everyone working on NQI is trying to solve one, the second, or both of these questions:
1. How will I extract the necessary information from the brain?
2. How will I send the necessary information to the brain?
The first concerns the brain's output - that is, the recording of what the neurons say. The second concerns the introduction of information into the natural flow of the brain, or changing this natural flow in some way - that is, stimulating neurons.
These two processes are constantly going on in your head. Right now, your eyes are performing a specific set of horizontal movements that allow you to read this sentence. It is the neurons in the brain that output information to the machine (your eyes), and the machine receives the command and responds. And when your eyes move in a certain way, photons from the screen penetrate your retina and stimulate neurons in the occipital lobe of your cortex, allowing the picture of the world to enter your consciousness. The picture then stimulates neurons in another part of your brain, which allows you to process the information in the picture and make sense of the sentence.
The input and output of information is what the neurons of the brain do. The entire NCI industry wants to join this process.
At first it seems that this is not such a difficult task. After all, the brain is just a jelly ball. And the cortex - the part of the brain that we want to add to our recording and stimulation - is just a napkin conveniently located on the outside of the brain, where it can be easily accessed. Inside the cortex are 20 billion neurons - 20 billion small transistors that could give us a whole new way of controlling our lives, health, and the world if we learn to work with them. Is it really that hard to understand them? Neurons are small, but we know how to split an atom. The diameter of a neuron is 100,000 times the size of an atom. If an atom were a lollipop, a neuron would be kilometers across - so we should definitely be able to work with such quantities. Right?
What is the problem?
On the one hand, these are the right thoughts, because they lead to progress in the field. We can really do it. But as soon as you begin to understand what is really going on in the brain, it immediately becomes obvious: this is the most difficult task for a person.
Therefore, before we talk about NCIs themselves, we need to carefully study what the people who create NCIs are doing. The best thing is to enlarge the brain 1000 times and see what happens.
Remember our comparison of the cerebral cortex to a napkin?
If we enlarge the bark napkin 1000 times - and it was about 48 centimeters on each side - it will now be two blocks long in Manhattan. It will take about 25 minutes to get around the perimeter. And the whole brain will be the size of Madison Square Garden.
Let's put it out in the city itself. I am sure that several hundred thousand people who live there will understand us.
I chose 1000x magnification for several reasons. One of them is that we can all instantly convert sizes in our head. Every millimeter of the actual brain has become a meter. In a world of neurons that is much smaller, every micron has become a millimeter that is easy to imagine. Secondly, the bark becomes "human" in size: 2 mm thickness is now 2 meters - like a tall person.
Thus, we can walk up to 29th Street, to the edge of our giant napkin, and it is easy to see what is happening in its two-meter thickness. For demonstration, let's pull out a cubic meter of our giant crust to examine it, see what happens in a typical cubic millimeter of real bark.
What do we see in this cubic meter? Meshanin. Let's clean it up and put it back.
First, let's place the somas - the small bodies of all the neurons that live in this cube.
Somas vary in size, but the neuroscientists I spoke with say that somas of neurons in the cortex are most often 10-15 microns in diameter (one micron = micron, 1/1000 millimeter). That is, if you put 7-10 of these in a line, this line will be the diameter of a person's hair. On our scale, the catfish will be 1-1.5 centimeters in diameter. Lollipop.
The volume of the entire crust fits into 500,000 cubic millimeters, and this space will contain about 20 billion soms. That is, the average cubic millimeter of the cortex contains about 40,000 neurons. That is, our cubic meter contains about 40,000 candies. If we divide our box into 40,000 cubes, each with a 3cm edge, each of our candy catfish will be in the center of its own 3cm cube, and all other catfish will be 3cm in all directions.
Are you here now? Can you imagine our meter cube with 40,000 floating candies?
Here is a microscopic image of a catfish in a real cortex; everything else around her has been removed:
Okay, so far it doesn't look that complicated. But the soma is only a tiny fraction of every neuron. From each of our lollipops extend twisted, branched dendrites that, on our scale, can stretch three to four meters in very different directions, and at the other end there may be an axon 100 meters long (if it crosses into another part of the cortex) or a kilometer (if it descends into the spinal cord and body). Each one is a millimeter thick, and these wires transform the bark into tightly woven electric vermicelli.
And there is a lot going on in this vermicelli. Each neuron has synaptic connections with 1,000 - sometimes up to 10,000 - other neurons. Since there are about 20 billion neurons in the cortex, this means there will be more than 20 trillion individual neural connections (and a quadrillion connections throughout the brain). Our cubic meter will have over 20 million synapses.
With all this, not only from each of the 40,000 lollipops in our cube there are thickets of vermicelli, but thousands of other spaghetti pass through our cube from other parts of the bark. And this means that if we tried to record signals or stimulate neurons specifically in this cubic region, we would have to be very difficult, because in the spaghetti jumble it would be difficult to determine which strands of spaghetti belong to our catfish candy (and God forbid, this paste will contain Purkinje cells).
And, of course, don't forget about neuroplasticity. The voltage of each neuron is constantly changing, hundreds of times per second. And tens of millions of synaptic connections in our cube will constantly change size, disappear and reappear.
But this is just the beginning.
It turns out that glial cells also exist in the brain - cells that come in many different types and perform many different functions, such as flushing out chemicals released at synapses, wrapping axons with myelin, and serving the brain's immune system. Here are some of the most common types of glial cells:
And how many glial cells are there in the cortex? About the same number as neurons. So add 40,000 more of these things to our cube.
Finally, there are blood vessels. Each cubic millimeter of cortex contains about a meter of tiny blood vessels. On our scale, this means that there is a kilometer of blood vessels in our cubic meter. This is how they look:
Digression on Connectoma
So, our meter box is packed, filled with electrified filling of varying complexity. Let's now remember that our box is actually a cubic millimeter in size.
Neurocomputer interface engineers need to either figure out what the microscopic catfish buried in this millimeter are saying, or stimulate certain catfish to do the right things. Good luck to them.
It would be difficult for us to do this with our 1000 times magnified brain. With a brain that perfectly turns into a napkin. But in reality he is not like that - this napkin lies on top of a brain full of folds (which, on our scale, are 5 to 30 meters deep). In fact, less than a third of the napkin cortex is on the surface of the brain - most of it lies in the folds.
In addition, there is not so much material with which it is possible to work in the laboratory. The brain is covered in many layers, including the skull - which at 1000x magnification would be 7 meters thick. And since most people do not really like it when their skull is open for too long - and indeed this is a dubious event - you have to work with tiny brain lollipops as carefully and delicately as possible.
And all this despite the fact that you are working with the bark - but a lot of interesting ideas on the topic of NCI deal with structures that are much lower, and if you stand on top of our city brain, they will lie at a depth of 50-100 meters.
Just imagine how much is going on in our cube - and this is just one 500,000th part of the cerebral cortex. If we broke our entire gigantic crust into equal meter cubes and lined them up, they would stretch for 500 kilometers - all the way to Boston. And if you decide to make a detour, which will take more than 100 hours while walking fast, at any time you can stop and look at the cube, and all this complexity will be inside it. All of this is now in your brain.
Elon Musk's Neuralink. Part 3: how happy you should be if you don't care about all this
Yours.
Back to part 3: flying over the nest of neurons
How will scientists and engineers deal with this situation?
They are trying to get the most out of the tools they currently have - the tools they use to record or stimulate neurons. Let's explore the options.
NCI tools
With what has already been done, three broad criteria can be distinguished by which the pros and cons of a recording instrument are assessed:
1) Scale - how many neurons can be recorded.
2) Resolution - how detailed the information the instrument receives - spatial (how closely your recordings tell which individual neurons are firing) and temporal (how well can you tell when the activity you are recording is occurring).
3) Invasiveness - whether surgery is necessary, and if so, how expensive.
The long-term goal is to collect the cream from all three and eat. But while the question inevitably arises, which of these criteria (one or two) can you neglect? The choice of this or that tool is not an increase or decrease in quality, it is a compromise.
Let's see what tools are currently in use:
fMRI
- Scale: large (shows information from around the brain)
- Resolution: low to medium - spatial, very low - temporal
- Invasiveness: non-invasive
fMRI is often used not in NCI, but as a classic recording tool - it gives you information about what is happening inside the brain.
fMRI uses MRI, a technology for magnetic resonance imaging. Invented in the 1970s, MRI was the evolution of X-ray CT scanning. Instead of X-rays, MRI uses magnetic fields (along with radio waves and other signals) to create images of the body and brain. Like this:
Complete set of cross-sections allowing you to see the entire head.
A very unusual technology.
fMRI ("functional" MRI) uses MRI technology to track changes in blood flow. What for? Because as areas of the brain become more active, they consume more energy, which means they need more oxygen - so blood flow increases in that area to deliver that oxygen. Here's what an fMRI scan can show:
Of course, there is always blood in the brain - this image shows where the blood flow has increased (red, orange, yellow) and where it has decreased (blue). And since fMRI can scan the entire brain, the results are three-dimensional:
FMRI has many medical uses, such as informing doctors about whether certain areas of the brain are functioning after a stroke, and fMRI has taught neuroscientists a lot about which areas of the brain are involved in these functions. The scan also provides important information about what is happening in the brain at a particular point in time, it is safe and non-invasive.
The big drawback is the resolution. fMRI scanning has literal resolution, like a computer screen pixels, only instead of two-dimensional, its resolution is represented by three-dimensional cubic volumetric pixels - voxels (voxels).
FMRI voxels have gotten smaller as technology has improved, resulting in increased spatial resolution. Voxels of modern fMRI can be as small as a cubic millimeter. The brain volume is about 1,200,000 mm3, so a high-resolution fMRI scan divides the brain into one million small cubes. The problem is that on a neural scale this is still quite a lot - each voxel contains tens of thousands of neurons. So, at its best, fMRI shows the average blood flow drawn in by each group of 40,000 neurons or so.
An even bigger problem is temporary resolution. fMRI monitors blood flow, which is inaccurate and occurs with a delay of about a second - an eternity in the world of neurons.
EEG
- Scale: high
- Resolution: very low spatially, medium-high temporal
- Invasiveness: non-invasive
Invented almost a century ago, EEG (electroencephalography) places many electrodes on the head. Like this:
EEG is definitely a technology that will look ridiculously primitive to humans in 2050, but at the moment it is one of the few instruments that can be used with completely non-invasive NCIs. An EEG records electrical activity in different areas of the brain, displaying the results as follows:
EEG charts can reveal information about medical problems such as epilepsy, track sleep patterns, or determine anesthesia dose status.
Unlike fMRI, EEG has a fairly good temporal resolution, receiving electrical signals from the brain as they appear - although the skull dilutes temporal accuracy considerably (bone is a poor conductor).
The main disadvantage is spatial resolution. EEG does not have it. Each electrode registers only the average value - the vector sum of charges from millions or billions of neurons (blurred because of the skull).
Imagine that the brain is a baseball stadium, its neurons are people in a crowd, and the information we want to receive will be, instead of electrical activity, a derivative of the vocal cords. In this case, the EEG will be a group of microphones outside the stadium, outside its outer walls. You will be able to hear when the crowd will start chanting and can even predict what they are about to shout about. You will be able to make out distinctive signals if there is a close fight or someone wins. You can also sort out if something out of the ordinary happens. That's all.
ECoG
- Scale: high
- Resolution: low spatial, high temporal
- Invasiveness: present
An ECoG (electrocorticography) is similar to an EEG in that it also uses electrodes on the surface - it just places them under the skull on the surface of the brain.
Dumb. But effective - much more effective than EEG. Without the interference from the skull, ECoG covers higher spatial (about 1 cm) and temporal resolutions (5 milliseconds). ECoG electrodes can be placed above or below the dura mater:
Layers on the left, from top to bottom: scalp, skull, dura mater, arachnoid, pia mater, cortex, white matter. Right signal source: EEG, ECoG, intraparenchymal (LFP, etc.)
Returning to the analogy with our stadium, ECoG microphones are located inside the stadium and closer to the crowd. Therefore, the sound will be much clearer than EEG microphones outside the stadium, and the EKoG will be able to distinguish between the sounds of individual segments of the crowd. But this improvement costs money - it requires invasive surgery. But by the standards of invasive surgery, this intervention is not all that bad. As one surgeon told me, “It is relatively non-invasive to place the filling under the dura. You have to poke a hole in your head, but it's not that scary."
Local field potential (LFP)
- Scale: small
- Resolution: medium-low spatial, high temporal
- Invasiveness: high
Let's move from surface electrode disks to microelectrodes - tiny needles that surgeons stick into the brain.
While some electrodes are still handcrafted today, new technologies use silicon wafers and manufacturing techniques borrowed from the integrated circuit industry.
The way local field potentials work is simple - you take one such ultra-thin needle with an electrode tip and insert it one or two millimeters into the cortex. There, it collects the average value of electrical charges from all neurons in a certain radius of the electrode.
LFP gives you not-so-bad spatial fMRI resolution combined with instant ECoG temporal resolution. By resolution standards, this is probably the best option out of all of the above.
Unfortunately, it is terrible in other ways.
Unlike fMRI, EEG and ECoG, the LFP microelectrode has no scale - it only tells you what the small sphere surrounding it is doing. And it is much more invasive as it actually enters the brain.
In a baseball stadium, the LFP is a single microphone hanging over one section of the seats, picking up a clear sound in that area and perhaps picking up a single voice here and there for a second or two - but for the most part, it senses a general vibration.
And a completely new development is a multi-electrode array, which is basically the idea of an LFP, only it consists of 100 LFPs at a time. The multi-electrode array looks like this:
A tiny 4 x 4 mm square with 100 silicon electrodes on it. Here's another one, here you can see how sharp the electrodes are - a few microns at the very tip:
Registration of individual units
- Scale: tiny
- Resolution: ultra high
- Invasiveness: very high
To record a wider LFP, the tip of the electrode is rounded slightly to give the electrode more surface area, and the resistance (incorrect technical term) is reduced to capture very weak signals from a wide range of locations. As a result, the electrode collects a chorus of activity from the local field.
Registration of individual units also involves a needle electrode, but their tips are made very sharp and resistance is also increased. Due to this, most of the noise is displaced and the electrode practically does not pick up anything until it is very close to the neuron (somewhere in 50 microns), and the signal from this neuron is strong enough to overcome the high-resistance electrode wall. Receiving separate signals from one neuron and having no background noise, this electrode can observe the private life of this neuron. Smallest possible scale, highest possible resolution.
Some electrodes want to take relationships to the next level and use the patch clamp method, which allows you to remove the tip of the electrode and leave only a tiny tube, a glass pipette, which will directly suck in the cell membrane of the neuron and take finer measurements.
Patch clamp also has this advantage: unlike all other methods, it physically touches the neuron and can not only record, but also stimulate the neuron by injecting current or maintaining the voltage at a certain level to perform specific tests (other methods can only stimulate entire groups whole neurons).
Finally, electrodes can completely subdue the neuron and actually penetrate the membrane to record. If the tip is sharp enough, it will not destroy the cell - the membrane is sealed around the electrode, and it will be very easy to stimulate the neuron or record the voltage difference between the external and internal environment of the neuron. But this is a short-term technique - a punctured neuron will not live long.
In our stadium, the registration of individual units will look like a unidirectional microphone attached to the collar of one fat man. Local potential clamping is a microphone in someone's throat that records the precise movement of the vocal cords. This is a great way to learn about a person's feelings about the game, but they will be taken out of context and will not be used to judge what is happening in the game or about the person himself.
That's all we have. At least that we use quite often. These tools are at the same time very advanced and will seem like Stone Age technologies to people of the future, who will not believe that we had to choose one of the technologies, to open the skull in order to get high-quality records of the brain.
But for all their limitations, these tools taught us a lot about the brain and led to the creation of the first curious brain-computer interfaces. More on them in the next part.
ILYA KHEL
Part One: The Human Colossus
Part Two: The Brain
Part Three: Flying Over the Nest of Neurons
Part four: neurocomputer interfaces
Part Five: The Neuaralink Problem
Part Six: Age of Wizards 1
Part Six: Age of Wizards 2
Part Seven: The Great Fusion