Global and local collaborative learning for no-reference omnidirectional image quality assessment

Abstract

Omnidirectional image (OI) has achieved tremendous success in virtual reality applications. With the continuous increase in network bandwidth, users can access massive OIs from the internet. It is crucial to evaluate the visual quality of distorted OIs to ensure a high-quality immersive experience for users. For most existing viewport based OI quality assessment(OIQA) methods, the inconsistent distortions in each viewport are always overlooked. Moreover, the loss of texture details brought by viewport downsampling procedure also limits the assessment performance. In order to address these challenges, this paper proposes a global-and-local collaborative learning method for no-reference OIQA. We adopt a dual-level learning architecture to collaboratively explore the non-uniform distortions and learn a sparse representation of each projected viewport. Specifically, we extract the hierarchical features from each viewport to align with the hierarchical perceptual progress of the human visual system (HVS). By aggregating with a Transformer encoder, the inconsistent spatial features in each viewport can be globally mined. To preserve more texture details during viewport downsampling process, we introduce a learnable patch selection paradigm. By learning the position preferences of local texture variations in each viewport, our method can derive a set of sparse image patches to sparsely represent the downsampled viewport. Comprehensive experiments illustrate the superiority of the proposed method on three publicly available databases. The code is available at https://github.com/ldyorchid/GLCNet-OIQA.