Cascade residual learning based adaptive feature aggregation for light field super-resolution

Abstract

Light field (LF) super-resolution aims to enhance the spatial or angular resolutions of LF images. Most existing methods tend to decompose 4D LF images into multiple 2D subspaces such as spatial, angular, and epipolar plane image (EPI) domains, and devote efforts to designing various feature extractors for each subspace domain. However, it remains challenging to select an effective multi-domain feature fusion strategy, including the fusion order and structure. To this end, this paper proposes an adaptive feature aggregation framework based on cascade residual learning, which can adaptively select feature aggregation strategies through learning rather than designed artificially. Specifically, we first employ three types of 2D feature extractors for spatial, angular, and EPI feature extraction, respectively. Then, an adaptive feature aggregation (AFA) module is designed to cascade these feature extractors through multi-level residual connections. This design enables the network to flexibly aggregate various subspace features without introducing additional parameters. We conduct comprehensive experiments on both real-world and synthetic LF datasets for light field spatial super-resolution (LFSSR) and light field angular super-resolution (LFASR). Quantitative and visual comparisons demonstrate that our model achieves state-of-the-art super-resolution (SR) performance. The code is available at https://github.com/haozhang25/AFA-LFSR.