Developers from Purdue University have created a neural network that can analyze fMRI images of the brain taken while watching a video, and then determine in real time what the person was watching.
The experiment involved three subjects, who were shown about a thousand small videos. During the show, the scientists were able to obtain a huge amount of fMRI data, which then began to demonstrate a convolutional neural network trained to match brain activity with the plots of the videos. Using the fMRI image, the network learned very quickly and correctly to determine what exactly the volunteer was watching at the time the image was taken.
In addition, the neural network learned to decipher the data of other people, based on the information obtained from fMRI of other volunteers, while the result was equally high both with fMRI data of healthy subjects and those with visual impairments.
Thanks to this study, scientists were able to decipher thoughts, and at the same time, they figured out which parts of the brain are responsible for recognizing images and videos. The point is that the brain divides the video into separate components. For example, if a person sees a car moving against the background of a wall, then one area of the brain recognizes the wall, and the other a car - thus, scientists were able to trace the work of the brain when comparing separate blocks of information and bringing it together into a single picture.
Viacheslav Larionov
