Efficient representation of disoccluded regions in 3D video coding

Abstract

Three-dimensional (3D) video technology has gained immense admiration in recent times due to its numerous applications, particularly in the television and cinema industry. Three-dimensional television (3DTV) and free-viewpoint television (FTV) are two well-known applications that provide the end-user with a real-world and high-quality 3D display. In both applications, multiple views captured from different viewpoints are rendered simultaneously to offer depth sensation to the viewer. A large number of views are needed to enable FTV. However, transmitting this massive amount of data is challenging due to bandwidth limitations. Multiview video-plus-depth (MVD) is the most popular format where in addition to color images, corresponding depth information is also available which represents the scene geometry. The MVD format with the help of depth image-based rendering (DIBR) enables the generation of views at novel viewpoints. In this paper, we introduce a panorama-based representation of MVD data with an efficient keyframe-based disocclusions handling technique. The panorama view for a stereo pair with depth is constructed from the left view and the novel appearing region of the right view which is not visible from the left viewpoint. The disocclusions that appear in the right view when obtained from the DIBR of the left view are collected in a special frame named as keyframe. On the decoder side, the left view is available with a simple crop of panorama view. The right view is obtained through DIBR of the left view combined with the appearing region from the panorama view. The disocclusions in this warped view are filled from the keyframe. The panorama view with additional keyframes and the corresponding depth map are compressed using the standard HEVC codec. The experimental evaluations performed on standard MVD sequences showed that the proposed scheme achieves excellent video quality while saving considerable bit rate compared to HEVC simulcast.