Research teams often experiment with video content using neural networks. Take NVIDIA, for example, which at the end of 2017 trained a neural network to change the weather and time of day on video. Another project of this kind was launched by researchers from Carnegie Melon University, who created a neural network for imposing facial expressions of one person on another's face.
The project is based on the DeepFakes technology for changing faces on video. It is based on a generative adversarial form of machine learning. Within its framework, the generative model tries to deceive the discriminatory and vice versa, as a result of which the system understands how the content can be transformed into a different style.
The cycle-GAN algorithm for transferring properties to another object is not ideal and allows for artifacts in the image. To improve the quality of the neural network, the researchers used its improved version of the Recycle-GAN. It takes into account not only the position of different parts of the face, but also the speed of their movement.
The neural network successfully transferred the facial expressions of TV presenter Stephen Colbert to the face of comedian John Oliver. Moreover, she transferred the flowering process of the daffodil to the hibiscus.
Researchers believe the technology could be used in cinematography. This will speed up the process and reduce the cost of making films. The ability of neural networks to change the weather on video will make it easier to teach electric cars to drive in different weather conditions.
Ramis Ganiev
Promotional video: