Two-dimensional hierarchical rate control scheme for light field compression using Multi-View Extension of High-Efficiency Video Coding

Abstract

Light field (LF) is envisioned to form a base for many visual applications, given its ability to capture a scene's spatial and angular information. However, using LF comes at the cost of increased bit rate and storage requirements. This paper performs a comprehensive analysis of the rate control aspect of LF data compression and proposes an extension of a rate control scheme originally designed for video signals. A two-dimensional hierarchical rate control strategy is proposed explicitly for LF data to meet a defined target bit rate with high accuracy while maintaining a similar rate-distortion efficiency as that presented in our previous state-of-the-art LF data compression scheme. Instead of considering the target bit budget for each video signal individually, the proposed rate control scheme considers the overall bit budget for an entire LF data compression in the bits allocation. To demonstrate the efficiency of the proposed scheme, the existing rate control scheme implemented in reference MV-HEVC, as well as the state-of-the-art LF coding methods on JPEG Pleno common test conditions, are considered for evaluation. The experimental results exhibit, on average, a 2 dB improvement in PSNR while achieving the target bit rate with 4.5% better accuracy than the reference MV-HEVC rate control scheme. It is clear from the extensive analysis that the proposed scheme outperforms the JPEG Pleno HEVC anchor scheme for all JPEG Pleno CTC lenslet LFs, thus fully justifying its inclusion in the JPEG Pleno Light Field Coding standard. In the future, the deep learning-based rate control scheme will be investigated, along with intra-frame bit allocation, to tackle low bit rates.