VPET: a toolset for collaborative virtual filmmaking

Abstract

Over the last decades the process of filmmaking has been subject to constant virtualization. Empty green screen stages leave the entire on-set crew clueless as real props are often replaced with virtual elements in later stages of production. With the development of virtual production workflows, solutions that enable the decision-makers to explore the virtually augmented reality have been introduced. However, current environments are either proprietary or lack usability, particularly when used by filmmakers without a specialized knowledge of computer graphics and 3D software. As part of the EU funded project Dreamspace, we have developed VPET (Virtual Production Editing Tool), a holistic approach for established film pipelines that allow on-set light, asset and animation editing via an intuitive interface. VPET is a tablet-based on-set editing application that works within a real-time virtual production environment. It is designed to run on mobile and head mounted devices (HMD), and communicates through a network interface with Digital Content Creation (DCC) tools and other VPET clients. The tool also provides functionality to interact with digital assets during a film production and synchronises changes within the film pipeline. This work represents a novel approach to interact collaboratively with film assets in real-time by maintaining fundamental parts of production pipelines. Our vision is to establish an on-set situation comparable to the early days of filmmaking where all creative decisions were made directly on set. Additionally, this will contribute to the democratisation of virtual production.