LV-BLiF: Harnessing Language-Vision Representation Learning for Blind Light Field Image Quality Assessment

Abstract

Light Field Images (LFIs) capture both angular and spatial information, enabling immersive visual experiences for observers. However, during acquisition and processing, LFIs often suffer from various degradations, accompanied by intricate spatio-angular distortions. Consequently, objective LFI Quality Assessment (LFIQA) has garnered significant research interest. Existing blind LFIQA methods predominantly operate in a content-agnostic manner. They primarily model low-level distortions while neglecting the semantic context, a limitation that significantly diverges from the perception mechanisms of the Human Visual System (HVS). To bridge the gap between artifact analysis and scene understanding, we propose LV-BLiF, a novel dual-branch LFIQA network based on language-vision representation learning. Firstly, the textual prompts-assisted branch utilizes a pre-trained quality-aware Large Multimodal Model (LMM) to extract deep semantic representations, which are transformed into semantic context vectors sensitive to spatial-angular quality variations via pseudo-label guided learning strategy. Subsequently, semantic quality scores are generated based on this high-level information. Secondly, the visual cues-assisted branch employs disentangling convolutions to decompose light field features into spatial, angular, and epipolar plane image subspaces. These features are processed by Light Field Residual Blocks (LFRBs) and adaptively integrated via a Mixture-of-Experts-like (MoE-like) gating mechanism to predict visual quality scores. Finally, the semantic and visual scores are fused through a learnable adaptive gating mechanism to derive the overall LFI quality score. Extensive experiments on benchmark datasets, including NBU, SHU, and Win5-LID, demonstrate that LV-BLiF significantly outperforms state-of-the-art methods and exhibits high consistency with human perception. The proposed metric will be publicly available at https://github.com/shuaibilx/LV-BLiF