Multi-Pass Renderer in MPEG Test Model for Immersive Video

Abstract

MPEG is developing an immersive video coding standard aims to deliver 6DoF (degrees of freedom) visual experience enabling motion parallax and omnidirectional viewing capabilities. A reference software has been made available to implement the immersive standard. To make it robust for various immersive content and deliver sharper synthesis with suppressed artifacts, a multi-pass add-on tool has been implemented for rendering at the decoder side. The tool runs view synthesis multiple times with different selection of views or patches of views in each pass based on the distance to the desired viewing position and orientation and merges the intermediate synthesized views to output a coherent and complete desired view. We share the multi-pass rendering results operating on whole views or on atlases (i.e. collection of patches from views) and compare them to the traditional single-pass synthesis results. We report significant subjective and objective improvements (1.7 ~ 4.2 dB gain in PSNR) for the multi-pass technique in the whole views case while having its synthesis results being atlas-dependent in the immersive coding case.