RDO-Based Light Field Image Coding Using Convolutional Neural Networks and Linear Approximation

Abstract

The increasing penetration of acquisition and display devices for Light Field (LF) content in the consumer market leads to the high proliferation of this new immersive media. This growing interest to LF images thus urgently raises the question of their compression. In this paper, we propose a convolutional neural networks (CNN)-based LF image coding scheme including both Rate Distortion Optimization (RDO) and post-processing steps. First, at the encoder side, the views are rearranged in sparse and dropped set of views. The former are compressed with a standard encoder and transmitted, while the dropped views are either linearly approximated or synthesized by a CNN using the encoded views as input. This choice is made on the basis of the proposed RDO process. At the decoder side, once the dropped views are either linearly approximated or synthesized by a CNN block, a post-processing step is performed to further enhance the quality of the reconstructed views. This post-processing block is based on superpixel to pixel-matching. Experimental results show that the proposed scheme provides views with high visual quality and overcomes the state-of-the-art LF image compression solutions by -30% in terms of BD-BR and 0.62 dB in BD-PSNR.