Adaptive Feature Transfer for Light Field Super-Resolution With Hybrid Lenses

Abstract

Reconstructing high-resolution (HR) light field (LF) images has shown considerable potential using hybrid lenses—a configuration comprising a central HR sensor and multiple side low-resolution (LR) sensors. Existing methods for super-resolving hybrid lenses LF images typically rely on patch matching or cross-resolution fusion with disparity-based rendering to leverage the high spatial sampling rate of the central view. However, the disparity-resolution gap between the HR central view and the LR side views poses a challenge for local high-frequency transfer. To address this, we introduce a novel framework with an adaptive feature transfer strategy. Specifically, we propose dynamically sampling and aggregating pixels from the HR central feature to effectively transfer high-frequency information to each LR view. The proposed strategy naturally adapts to different disparities and image structures, facilitating information propagation. Additionally, to refine the intermediate LF feature and promote angular consistency, we introduce a spatial-angular cross attention block that enhances domain-specific feature by appropriate weights generated from cross-domain feature. Extensive experimental results demonstrate the superiority of our proposed method over state-of-the-art approaches on both simulated and real-world datasets. The performance gain has significant potential to facilitate the down-stream LF-based applications.