MetaCineMoji: Visualizing film set communication in an interactive interface for collaboration in virtual LED production

Abstract

LED-VP is now mainstream in high-end studios, yet the capital cost of real-time volume stages renders training with this technology prohibitively expensive. Most film institutions simply cannot afford to build one LED-VP set for their students. This paper explores an alternative solution to the training of LED-VP by developing a virtual counterpart. However, significant challenges arise in simulating a virtual environment where students can operate in an LED-VP. Our study proposes the virtual environment of LED-VP, a virtual reality collaborative learning (VRCL) system. We developed an interface featuring visual symbols, namely MetaCineMoji, for film operation to facilitate smooth communication and learning processes in LED-VP workflows. MetaCineMoji demonstrates the feasibility of multi-person collaborative learning in a virtual filming studio, translating film set communication to visual symbols for lighting design, scene construction, and collaborative work with key stakeholders, e.g., directors, cinematographers, and gaffers. We explore the impact of the film-operation interfaces containing visual symbols on social interaction factors within the virtual studio. We conducted evaluations with 24 participants. Our findings show that our system equipped with the film-operation interface, enabled by visual symbols, significantly enhances social interaction among learners and results in significantly higher learning outcomes than systems without the interface.