The ability to control thoughts in one form or another has been widely used by the authors of numerous science fiction novels. But recently, visualization of mental images has ceased to belong to the realm of fantasy.
In the early 2000s, fMRI was used to make the first attempts at "reverse retinotopy" (retinotopy is an ordered projection of the retina onto the visual area of the cerebral cortex). At first, the attempts were rather timid: the subjects were shown images and simultaneously took data on the activity of various areas of the brain using fMRI. Having collected the necessary statistics, the researchers tried to solve the inverse problem - to guess what a person is looking at using the map of brain activity.
In simple pictures, where the main role was played by spatial orientation, the location of objects or their category, everything worked quite well, but it was still very far from "technical telepathy". But in 2008, scientists from the Institute of Neurosciences at the University of California at Berkeley, led by psychology professor Jack Gallant, tried to do this trick with photographs. They divided the studied area of the brain into small elements - voxels (3D elements) - and tracked their activity while the subjects (in their role were played by two authors of the work) were shown 1,750 different photographs.
Based on this data, the scientists built a computer model, which they "trained" by showing 1000 other photographs and receiving 1000 different voxel activation patterns as output. It turned out that by showing the same 1000 photographs to the subjects and comparing the patterns taken from their brains with those predicted by the computer, it is possible with a fairly high accuracy (up to 82%) to determine which photograph a person is looking at.
Promotional video:
Moving pictures
In 2011, a team of researchers led by the same professor Gallant from the University of California at Berkeley achieved significantly more interesting results. By showing subjects 7,200 seconds of "training" movie clips, the researchers studied the activity of multiple brain voxels using fMRI. But here they are faced with a serious problem: fMRI reacts to the absorption of oxygen by brain tissues - hemodynamics, which is a much slower process than changes in nerve signals. It doesn't really matter for studying reaction to still images - a photo can be shown for a few seconds, but with dynamic videos, serious problems arise. Therefore, scientists have created a two-stage model,which connects slow hemodynamics and fast neural processes of visual perception.
After building an initial computer model of the brain's "response" to various videos, the researchers trained it using 18 million one-second videos randomly selected from YouTube. Then the subjects were shown "test" films (other than "training" ones), studying the brain activity using fMRI, and the computer selected from these 18 million one hundred clips that caused the closest pattern of activity, after which it averaged the image on these clips and produced the "average result". The correlation (coincidence) between the image that the person sees and the one that is generated by the computer was about 30%. But for the first "mind reading" this is a very good result.
Sleep in hand
But the accomplishments of Japanese researchers at the Neuroscience Laboratory of the Telecommunications Research Institute in Kyoto, the Science and Technology Institute in Nara, and the National Institute of Information and Communication Technology in Kyoto appear to be far more significant. In May 2013, they published Neural Decoding of Visual Images during Sleep in Science. Yes, scientists have learned to dream. More precisely, not to see, but to spy!
There are several ways to "see" what is happening in the brain of a living person. Electroencephalography (EEG) uses measurements of weak electrical potentials at the surface of the scalp, while magnetoencephalography (MEG) records very weak magnetic fields. These methods allow you to track the total electrical activity of the brain with a high temporal resolution (units of milliseconds). Positron emission tomography (PET) allows you to see the activity of specific areas of the working brain by tracking previously injected substances containing radioactive isotopes. The method of functional magnetic resonance imaging (fMRI) is based on the fact that oxyhemoglobin in the blood that carries oxygen to tissues differs in its magnetic properties from deoxyhemoglobin that has already given up oxygen. FMRI can be used to see the active areas of the brainoxygen absorbing. The spatial resolution of this method is millimeters, and the temporal - of the order of fractions of a second.
Recording signals of brain activity using fMRI, three subjects were awakened (about 200 times) at stages of shallow sleep and asked to describe the content of the last dream. Key categories were identified from the reports, which, using the WordNet lexical database, were combined into groups of semantically similar terms (synsets), organized into hierarchical structures. FMRI data (nine seconds before waking up) were sorted by synset. To train the recognition model, awake subjects were shown images from the ImageNet database corresponding to synsets, and a map of brain activity in the visual cortex was studied. After that, the computer was able to predict with a probability of 60-70% what a person sees in a dream based on the activity of various brain regions. This, incidentally, indicates thatthat a person dreams using the same areas of the visual cortex that are used for normal waking vision. That's just why we see dreams at all, scientists cannot yet say.
Dmitry Mamontov