Perhaps, if you are ever going to watch a movie, the video streaming service algorithm will recommend a blockbuster that is scripted by artificial intelligence (AI), robots are actors, and animation and rendering are done by deep learning. It is even possible that the script for this film was read by an AI and advised the studio to purchase it.
It is easy to imagine how algorithms and robots will send the film industry to the same place as industrial and service workers, and that artistic filmmaking is in its dying stage. However, this does not apply to the film industry, on the contrary - AI enhances the creative possibilities of Hollywood, rather than suppress them.
It is true that some professions and occupations are becoming obsolete as computers get better at handling them. A VFX specialist no longer needs to wear a beret and be able to paint a backdrop - studios need engineers who are adept at training deep learning algorithms to perform routine work such as refining a video effect or making a digital character appear real. That said, creative workers spend less time on the computer doing thorough frame-by-frame editing and can do more interesting things, explains Darren Hendler, Digital Domain Leader.
Just as computers have eliminated the need for animators to manually draw each frame, advanced algorithms can automatically apply complex video effects. In either case, animators do not remain without work.
“We found that there are so many mundane tasks available to AI that can complete them much faster, freeing up time for people to be creative,” Hendler says. - I think there will be a lot of technology to make it easier for actors to present fantastic characters. Increasingly, the actors playing in group scenes will then be replaced in those scenes with computer generated avatars.”
Recently, Handler's team used AI and other sophisticated software to transform Josh Brolin into Thanos in Avengers: Infinity War. More precisely, they used an AI algorithm, trained on high-resolution scans of Brolin's face, to track his facial expressions down to wrinkles, and then using another algorithm superimposed the rendered face on Thanos' body.
This approach allows you to combine the best - the high resolution obtained with a movie camera, and more subtle acting as a result of the actor's work surrounded by other actors, and not alone in front of a green screen. And while facial blending usually takes weeks, Digital Domain's machine learning algorithm handles this in near real-time, creating a digital mirror for Brolin.
“On the very first day of filming, Brolin could see how his character would look and act,” says Hendler.
Promotional video:
Yes, these algorithms are able to quickly cope with tasks that required teams of specialists to complete. When used effectively, they can provide the best performance, precise editing, and the most advanced video effects available today. And while the most advanced (that is, expensive) algorithms have so far been limited to use only in Disney studio blockbusters, Handler believes they will become the norm over time.
“I think it will find widespread use. The main obstacle is that it requires a completely different approach and course of action, and people are still figuring out how to make it work. But as humans gain experience in training machines to perform animation tasks, it is logical to expect more and more areas in filmmaking where AI will be applied.
“Machine learning has yet to be widely adopted because producers don't have a good understanding of the technology yet,” says Hendler. “But we're already seeing deep and machine learning penetrate very specific areas. This is very new and different from anything we have done before."
Many such applications have already appeared. In January, Kristen Stewart (yes, Twilight's Bella Swan) directed a short film in partnership with Adobe to create a neural network that makes footage look like Stewart's impressionist paintings.
Fine-tuning the u parameter changes the degree to which the film resembles an impressionist painting.
Meanwhile, the Disney studio has created acrobatic robots that can be thrown high in the air, then edited (possibly with the help of AI) to make them look like actors. Now the actors can relax and concentrate on less dangerous episodes.
But soon, AI in filmmaking will be able to move beyond the realms of acting and editing - it will be able to participate in the decision about the production of the film. The Belgian company Scriptbook has created an algorithm that it claims can predict, by analyzing a script, whether a film will be commercially successful.
Usually the work with the script is carried out by the film studio - assistants and trainees, and the latter does not even have to be paid under California law. To justify spending $ 5,000 against the often unpaid work of people, the company claims its algorithm is three times better at predicting box office success than people reading a script. The company also says it would not recommend Sony Pictures to shoot 22 of their most notable box office flops in the past three years, which would have saved the company millions of dollars.
The Scriptbook does not yet answer questions regarding their technology and the limitations of the algorithm, but AI is known to be able to predict the MPAA rating (age limit), determine who the characters are and what emotions they express, and outline the target audience of the script. He can also determine if the script passes the Beckdel test - the minimum requirement for the representation of women in the works. The algorithm is also able to determine if the cast of characters in the film is sufficiently diverse, although it should be noted that many scripts do not mention the character's race.
Of course, all of this can be done by people reading the script. And a human-written report on the script includes a detailed summary and recommendations whether a particular studio, depending on its specifics and audience, should take this script to work. Given the emotional complexities of an AI, it is unlikely that a Scriptbook will provide the same level of in-depth analysis as a human. Still, the company itself suggested using the system to help reviewers of scripts, and not as a replacement.
The algorithm will help people base their scenario analysis on emotionless, reliable data. While it might seem like a lot of human creativity is needed here, tools like the Scriptbook can help studios make better financial decisions.
The automated future of Hollywood does not mean the elimination of a person from the process (unless such a decision will be made by those in power). Rather, Hendler looks forward to a future in which artists continue to create, working in conjunction with time-saving machines that do routine and simplify work.
“We're still working out ways to apply machine learning to a variety of problems,” says Hendler. "In the next two to three years, we expect a sharp increase in the speed of operations in the field of video effects, as well as an improvement in quality."
Vadim Tarabarko