User virtual costume visualisation in an augmented virtuality immersive cinematic environment

Recent development of affordable head-mounted displays (HMDs) has led to accessible Virtual Reality (VR) solutions for watching 360-degree panoramic movies. While conventionally users cannot see their body while watching 360 movies, our prior work seamlessly blended a user's physical body into a 360 virtual movie scene. This paper extends this work by overlaying context-matching virtual costumes onto the user's real body. A prototype was developed using a pair of depth cameras and an HMD to capture the user's real body and embed it into a virtual 360 movie scene. Virtual costumes related to the movie scene are then overlaid on user's real body to enhance the user experience. Results from a user study showed that augmenting the user's real body with context-matching virtual costume was most preferred by users while having no significant effect on sense of presence compared to showing only user's body in a 360 movie scene. The results offer a future direction to generate enhanced 360 VR movie watching experiences.