Estimating the Quality of Experience of Immersive Contents

Abstract

Recent technology advancements have driven the production of plenoptic devices that capture and display visual contents, not only as texture information (as in 2D images) but also as 3D texture-geometric information. These devices represent the visual information using an approximation of the plenoptic illumination function that can describe visible objects from any point in the 3D space. Depending on the capturing device, this approximation can correspond to holograms, light fields, or point clouds imaging formats. Naturally, the success of immersive applications depends on the acceptability of these formats by the final user, which ultimately depends on the perceived quality of experience. Several subjective experiments have been performed with the goal of understanding how humans perceive immersive media in 6 Degree-of-Freedom (6DoF) environments and what are the impacts of different rendering and compression techniques on the perceived visual quality. In this context, an open area of research is the design of objective methods that estimate the quality of this type of content. In this talk, I describe a set of objective methods designed to estimate the quality of immersive visual contents -- an important aspect of the overall user quality of experience. The methods use different techniques, from texture operators to convolution neural networks, to estimate quality by also taking into consideration the specificities of the different formats. Finally, I discuss some of the exciting research challenges in the area of realistic immersive multimedia applications.