Lost in light field compression: Understanding the unseen pitfalls in computer vision

Abstract

Could we be overlooking a fundamental aspect of light fields in our quest for efficient compression? The vast amount of data enclosed in a light field makes compression a necessity. Yet, from an application point of view, the focus is predominantly on visual consumption while light fields have properties that can potentially be used in various other tasks. This paper examines the impact of light field compression on the performance of subsequent computer vision tasks. We investigate the variations in quality across perspectives and their impact on face recognition systems and disparity estimation. By leveraging a diverse dataset of light field images, we thoroughly evaluate the performance of various face recognition algorithms when subjected to different conventional and learning-based compression techniques, such as JPEG Pleno, ALVC, and SADN-QVRF. Our findings reveal a noticeable decline in peak recognition performance as compression levels increase, given specific recognition frameworks. Furthermore, we identify a significant shift in the recognition threshold, particularly in response to higher degrees of compression. Secondly, by relying on a novel disparity estimation algorithm, we explore the loss of information across light field perspectives. Our results highlight a disconnect between the preservation of visual fidelity and the loss of minute detail crucial for the preservation of disparity information in light field images. The findings presented herein aim to contribute to the development of efficient compression strategies while emphasizing the delicate balance between compression efficiency, subjective quality, and feature preservation with the aim of increased accuracy in specialized light field systems.