VFX Supervisors Andrew Whitehurst and Robert Weaver of Lucasfilm’s Industrial Light and Magic have revealed that machine learning was essential to the de-aging scenes in Indiana Jones and the Dial of Destiny.
WARNING: The following article contains spoilers for Indiana Jones and the Dial of Destiny.
In an interview with The Hollywood Reporter, the leaders of the visual effects team opened up about the role machine learning played in rendering the ground-breaking 2,350 VFX shots depicting a youthful Harrison Ford as Indiana Jones for scenes in the fifth installment of the film franchise. Weaver explained:
“There are absolutely components to it that we rely on. Definitely. And that’s part of the evolution of us doing more successful digital human performance. [To teach the computer] we primarily relied on the older Indy films and that target age. But we also captured present day.”
Whitehurst broke down the process of training the machine learning model to recognize and reconstruct three-dimensional images of Ford as a younger man, with a combination of film and photos from the past and performances by the legendary actor in the present. The VFX team relied on ILM Face Swap technology, which combines machine learning and CG assets.
“We shot a lot of reference material, as well as the actual shots from the film we were making. We went and deliberately set up the cameras and got Harrison to perform as he is now. We learn, the computer learns, but I think it’s also worth saying that even within feeding the footage, you need to be careful and select your footage, because some footage might skew results in a way that actually isn’t helpful for a particular shot. So there’s a lot of kind of skill and judgment in terms of selecting what you are going to use as reference.”
The team painstakingly compiled material of Ford at the desired age for the scene and used it to create photorealistic renderings of Jones for high-octane action sequences, including breathtaking chase scenes on a train and in a tuk-tuk. According to Weaver:
“That curation process that Andrew was just talking about was incredibly important. Artists would make determinations as to what is helpful.”
Some might say that Whitehurst and Weaver seem oblivious to the fact that fans and critics alike heaped criticism on the de-aging scenes in the film. The reality is that today’s moviegoers are well versed in de-aging technology through products like Reface: Face-Swap App and AI Avatar Generator and FaceApp, which are downloadable on any mobile device. As a result, audiences have very exacting visual standards and are quick to spot some of the more clumsily rendered images, as in the case of Ford’s younger doppelganger.
Nevertheless, Indiana Jones and the Dial of Destiny is a cinematic milestone that ushers in a new era for creatives to explore artificial intelligence technology as they traverse new filmmaking frontiers.