Towards Distributed and Interactive Multi-cam and Multi-device VR360 Video Experiences

Abstract

The production and consumption of multimedia content is continuously increasing, and this particularly affects to immersive formats, like VR360 video. Even though significant advances have been witnessed with regard to the processing, delivery and consumption of interactive VR360 video, key challenges and research questions still need to be addressed to efficiently provide interactive multi-camera and multi-user VR360 video services over distributed and heterogeneous environments. This research work aims at providing novel and efficient contributions to overcome existing limitations in this topic. First, it will develop an end-to-end modular web-based VR360 video platform, including the measurement of Quality of Service (QoS) and activity metrics, to be used as a research testbed. Second, it will provide lightweight yet efficient viewport-aware video processing and delivery strategies to dynamically concentrate the video resolution on the user’s viewport, with a single stream and decoding process through the web browser. Third, it will propose innovative encoding, signaling and synchronization solutions to enable an effective support for multi-camera and multi-device VR360 services, in a synchronized manner and with the lowest latency possible. Fourth, it will explore how to effectively provide social viewing scenarios between remote users while watching the same or related VR360 videos, assisted with interactive and guiding techniques.