Stagecraft: the story on HearLore

Stagecraft

The 15th of November 2019 marked the beginning of a quiet revolution in Hollywood when the first episode of The Mandalorian aired, featuring a technology that would soon redefine how movies are made. Before this moment, creating a galaxy far, far away required actors to stare at green screens in empty soundstages, with their surroundings added weeks or months later by computer artists. The new approach placed actors inside a massive, curved video wall that displayed photorealistic environments in real time, allowing them to see the world they were inhabiting while filming. This innovation, known as StageCraft, was not merely a new tool but a fundamental shift in the relationship between the physical and digital worlds of filmmaking. The technology was developed by Industrial Light & Magic, the visual effects division of Lucasfilm, to solve a specific problem that had plagued the industry for decades. Cinematographer Greig Fraser encountered multiple issues while shooting the film Rogue One in 2016, which inspired the idea to use large LED screens as a component of the set. These issues included the difficulty of matching lighting on actors to the digital backgrounds and the lack of depth that green screens provided. The idea was further developed by a team including Industrial Light & Magic's Richard Bluff and Rob Bredow, as well as Kim Libreri of Epic Games. When director Jon Favreau began work on the Disney+ series The Mandalorian, ILM found a perfect opportunity to use the technology with a director prepared to use it. The result was a facility known as The Volume, a soundstage where live-action actors and sets were surrounded by large, very high-definition LED video walls. These walls displayed computer-generated imagery backdrops, once traditionally composited primarily in post-production after shooting with chroma key screens. When shooting, the production team was able to realign the background instantly based on moving camera positions. The entire CGI background could be manipulated in real time, giving actors and directors immediate feedback on how their scenes would look in the final product.

The Engine Behind The Illusion

At the heart of StageCraft lies a piece of software originally designed for video games, not cinema. ILM used Epic Games' Unreal Engine, a popular game engine, to handle real-time 3-D rendering of computer-generated imagery environments. This choice was significant because it allowed for the creation of complex, interactive worlds that could be rendered instantly as the camera moved, rather than waiting for hours of processing time. Other technology partners in StageCraft include FuseFX, Lux Machina, Profile Studios, Nvidia, and ARRI, each contributing specialized hardware and software to the ecosystem. The integration of these technologies required a level of coordination that had never been attempted before in the film industry. In September 2020, it was announced that a second permanent volume was being created at Manhattan Beach Studios in Los Angeles, in addition to the first built for The Mandalorian, which was expected to be completed in March 2021. One at Pinewood Studios in London, to open in February 2021, and a larger, custom one at Fox Studios Australia followed. These new volumes would be larger, use more LED panels, and offer higher resolution than the original Manhattan Beach one. ILM also has the ability to provide pop up virtual production configurations in other locations. A volume was announced to be built in Vancouver in November 2021, and was planned to be opened in early 2022. The technology continued to evolve rapidly, with ILM iterating the technology to StageCraft 2.0 for the second season of The Mandalorian. This version featured a larger volume as well as more specialized software. One example of this software is Helios, a rendering engine designed by ILM specifically for StageCraft hardware. Helios allowed for even more complex lighting and shadow interactions, making the digital environments indistinguishable from real-world locations. The collaboration between game developers and film makers created a new language of visual storytelling, where the boundaries between pre-production, production, and post-production began to blur. The ability to see the final image on set meant that directors could make creative decisions in the moment, rather than hoping their vision would translate correctly after weeks of digital compositing. This shift empowered filmmakers to take risks and explore new visual styles that were previously too time-consuming or expensive to attempt.

What is StageCraft and who developed it?

StageCraft is a visual effects technology developed by Industrial Light & Magic that uses large LED video walls to display photorealistic environments in real time. The technology was created to solve lighting and depth issues associated with traditional green screen filmmaking.

When did StageCraft first appear in a major production?

The first episode of The Mandalorian aired on the 15th of November 2019 and marked the beginning of the public use of StageCraft technology. The system was first tested during the development of The Lion King in 2019 before its full implementation in The Mandalorian.

Which software powers the real-time rendering in StageCraft?

ILM uses Epic Games Unreal Engine to handle real-time 3-D rendering of computer-generated imagery environments within StageCraft. A specialized rendering engine named Helios was also designed by ILM specifically for StageCraft hardware to manage complex lighting and shadow interactions.

Where are the permanent StageCraft volumes located?

Permanent StageCraft volumes are located at Manhattan Beach Studios in Los Angeles, Pinewood Studios in London, and Fox Studios Australia. A volume was announced to be built in Vancouver in November 2021 and was planned to be opened in early 2022.

How does StageCraft change actor performance compared to green screens?

StageCraft allows actors to see the world they are inhabiting in real time while filming, providing tangible environments that react to their movements. The lighting on the LED walls reflects onto the actors, creating a natural interaction between the subject and the background that is impossible to achieve with green screens.

From Green Screens To Living Worlds

The transition from traditional green screen techniques to the immersive environment of The Volume represented a paradigm shift in how actors interacted with their surroundings. Before StageCraft, actors often had to imagine the world around them, relying on the director's description or a simple marker on the floor. The new technology provided a tangible environment that reacted to the actors' movements and the camera's position. This change had a profound impact on the performances, as actors could now see the lighting and textures of their environment in real time. The lighting on the LED walls also reflected onto the actors, creating a natural interaction between the subject and the background that was impossible to achieve with green screens. This realism extended to the way the camera moved through the scene, as the background would shift and perspective would change dynamically, mimicking the behavior of a real physical location. The technology was first tested during the development of The Lion King in 2019, when director Jon Favreau worked with ILM to develop technologies in order to better visualize shots within a CGI space. This experience laid the groundwork for the more advanced systems used in The Mandalorian. The success of The Mandalorian led to a wave of productions adopting the technology, including The Book of Boba Fett, How I Met Your Father, Our Flag Means Death, Obi-Wan Kenobi, Andor, House of the Dragon, Ahsoka, Percy Jackson and the Olympians, Avatar: The Last Airbender, Doctor Who series 14, and Star Wars: Skeleton Crew. Feature films such as In Vacanza su Marte, The Midnight Sky, The Batman, Thor: Love and Thunder, Black Adam, The Fabelmans, Ant-Man and the Wasp: Quantumania, The Creator, The Marvels, Daddio, Transformers One, A Big Bold Beautiful Journey, The Lost Bus, and Tron: Ares also utilized StageCraft. The technology proved so versatile that it was used for everything from epic space battles to intimate dramas, demonstrating its adaptability across genres. The ability to create complex environments on set allowed filmmakers to shoot scenes that would have been impossible to film in the real world, such as the tornado sequence from Steven Spielberg's The Fabelmans, which was filmed on the StageCraft Volume at Manhattan Beach Studios. The finished scene showcased the seamless integration of practical and digital elements, proving that the technology could handle even the most challenging visual effects sequences.

The Global Expansion Of The Volume

The success of The Mandalorian sparked a global expansion of StageCraft, with new volumes being built in major film production hubs around the world. In September 2020, it was announced that a second permanent volume was being created at Manhattan Beach Studios in Los Angeles, in addition to the first built for The Mandalorian, which was expected to be completed in March 2021. One at Pinewood Studios in London, to open in February 2021, and a larger, custom one at Fox Studios Australia followed. These new volumes would be larger, use more LED panels, and offer higher resolution than the original Manhattan Beach one. ILM also has the ability to provide pop up virtual production configurations in other locations. A volume was announced to be built in Vancouver in November 2021, and was planned to be opened in early 2022. The expansion of StageCraft was not limited to the United States and the United Kingdom, as the technology was adopted by production companies in Australia, Canada, and beyond. The global reach of StageCraft allowed filmmakers to shoot complex scenes in multiple locations without the need for expensive travel or the logistical challenges of building physical sets. The technology also enabled remote collaboration, with directors and cinematographers able to work with teams in different parts of the world to create cohesive visual styles. The ability to create custom volumes for specific productions meant that filmmakers could tailor the technology to their unique needs, whether they required a small, intimate set or a massive, expansive environment. The flexibility of StageCraft allowed for the creation of diverse worlds, from the gritty streets of Gotham City in The Batman to the fantastical landscapes of House of the Dragon. The technology also facilitated the production of high-budget films with tight schedules, as the ability to shoot complex scenes on set reduced the time needed for post-production. The global expansion of StageCraft was a testament to the technology's versatility and the growing demand for immersive, high-quality visual effects in modern filmmaking. The success of The Mandalorian and the subsequent adoption of StageCraft by other productions demonstrated the potential of the technology to transform the industry, making it a standard tool for filmmakers around the world.