News from the category of events that happen, perhaps, no more often than events of the level of the Higgs boson discovery. Popular Hollywood actress Kristen Stewart wowed the entire AI research community by co-authoring a published scientific paper on machine learning.
The Dream of Millions of Twilight fans recently made its directorial debut and directed the short drama Come Swim, which used machine learning technology known as style translation. It is a technique where the aesthetics of one image or video frame is superimposed on the aesthetics of another image or frame to create an impressionistic visual style.
With the support of visual effects specialist Bhautik Yoshi and producer David Shapiro, Stewart co-authored a scientific paper describing the process of making the film and the technologies used in it. The article was published in the largest scientific online library arXiv, where publications are not peer-reviewed.
Fans of the actress, as well as experts in the field of artificial intelligence research, were surprised (and pleased) to learn that Stewart was actively involved in this work.
The work itself is titled “Bringing Impressionist Paintings to Come Swim to Life Using Neural Network-Driven Style Transfer” and offers a detailed analysis of how a similar machine learning approach can be used to create movies. In the article itself, the short film Let's Go Swim is described as "a poetic impressionist portrait of a grief-stricken man under water."
To transfer the visual style between frames, the filmmaking team used existing neural networks, with the help of which they first transferred the desired style to the test frame, and then changed the picture step by step, adding "blocks of new colors and textures" until they got the desired Effect. After adjusting the style transfer as needed, they applied this method to various parts of the painting, creating frames like the ones below. Basically, this method is similar to some very advanced progressive scan.
Of course, the expected negative reaction to the published work followed: "Why the hell is a Hollywood actress even getting into a machine learning environment where she doesn't understand a damn thing?" However, more important here is the fact that machine learning tools, which were once interesting only to those people who practically lived with them, are becoming more and more popular. Open source structural AI like the same Tensor Flow and Keras allow anyone to try their hand at writing AI code and creating specific commercial processing methods based on them, like the same style transfer (by the way, the same social network Facebook very actively uses image filters created by based on this method) and contribute to the promotion of such technologies in culture.
Promotional video:
Ultimately, the AI revolution will depend on more than massive amounts of data and powerful processors to classify and adapt that data to work. It is also important here to create an open community of developers, as well as accessible tools for work. And Stewart's research paper is a great example of how this can work and what we can achieve through it.
The short film Let's Go Swim was shown at the 2017 Sundance Film Festival. Below you can watch a teaser of the film.
NIKOLAY KHIZHNYAK