Light field spatial super-resolution network based on spatial-angular asymmetry and non-local correlation

Abstract

In recent years, deep neural networks (DNNs) have made significant progress in the spatial super-resolution (SR) of light field (LF) images. However, existing methods fail to fully account for spatial-angular asymmetry and non-local correlation. As a result, it is challenging to fully exploit the rich spatial-angular features during LF image feature extraction. To address these issues, we propose a novel LF spatial SR network. Specifically, due to spatial-angular asymmetry, we design spatial feature extractor (SFE) and angular feature extractor (AFE) with different receptive fields. This asymmetrical design enables comprehensive extraction of rich spatial information and reduces angular information redundancy when processing sub-aperture images (SAI) and macro-pixel images (MacPI). Furthermore, based on spatial-angular non-local correlation, we propose an epipolar feature extractor (EFE) to deeply extract long-range spatial-angular feature information from the epipolar-plane image (EPI). Moreover, we integrate SFE, AFE, and EFE into a multi-dimensional feature extraction module (MDFEM) to efficiently process SAI, MacPI, and EPI. By cascading multiple MDFEMs, the proposed network can deeply explore the spatial and angular information of the LF. Experimental results on both real-world and synthetic LF datasets show that the proposed method outperforms state-of-the-art methods in terms of visual quality and quantitative metrics, and can be more effectively applied to LF tasks such as depth estimation and LF refocusing.