MLANet: Multilevel aggregation network for binocular eye-fixation prediction

Abstract

Saliency prediction is an underexplored but fundamental task in computer vision, especially for binocular images. For binocular images, saliency prediction aims to find the most visually distinctive parts to imitate the operation of the human visual system. Existing saliency prediction models do not utilize the extracted multilevel features adequately. In this paper, we introduce a novel saliency prediction framework called multilevel aggregation network (MLANet) to explicitly model the multilevel features of binocular images. We split the multilevel features into shallow and deep features using the ResNet-34 backbone. We consider the first three and last three stage features as shallow and deep features, respectively for saliency prediction reconstruction. First, we introduce the aggregation module to integrate adjacent shallow and deep features. Thereafter, we utilize shallow aggregation and multiscale-aware modules to locate objects of different sizes with features from adjacent levels. For better reconstruction, we integrate deep- and shallow-level features with the help of feature guidance and deploy dual-attention modules to select the discriminative enhanced characteristics. Experimental results on two benchmark datasets, NCTU (CC of 0.8575, KLDiv of 0.2648, AUC of 0.8856, and NSS of 2.0138) and S3D (CC of 0.7977, KLDiv of 0.2024, AUC of 0.7954, and NSS of 1.2963), for binocular eye-fixation prediction show that the proposed MLANet outperforms the other compared methods.