BPGI: A Brain-Perception Guided Interactive Network for Stereoscopic Omnidirectional Image Quality Assessment

Abstract

Stereoscopic omnidirectional image quality assessment is a combination task of stereoscopic image quality assessment and omnidirectional image quality assessment, which is more challenging than traditional three-dimensional images. Previous works fail to present a satisfying performance due to neglecting human brain perception mechanism. To solve the above problem, we proposed an effective brain-perception guided interactive network for stereoscopic omnidirectional image quality assessment (BPGI), which is built following three perception steps: visual information processing, feature fusion cognition, and quality evaluation. Considering the stereoscopic perception characteristics, binocular and monocular visual features are both extracted. Following human complex cognition mechanism, a Bi-LSTM module is introduced to dig the deeply inherent relationship between monocular and binocular visual feature and improve the feature representation ability of the proposed model. Then a visual feature fusion module is built to obtain effective interactive fusion for quality prediction. Experimental results prove that the proposed model outperforms many state-of-the-art models, and can be effectively applied to predict the quality of stereoscopic omnidirectional images.