Simple baselines can fool 360° saliency metrics

Abstract

Evaluating a model’s capacity to predict human fixations in 360° scenes is a challenging task. 360° saliency requires different assumptions compared to 2D as a result of the way the saliency maps are collected and pre-processed to account for the difference in statistical bias (Equator vs Center bias). However, the same classical metrics from the 2D saliency literature are typically used to evaluate 360° models. In this paper, we show that a simple constant predictor, i.e. the average map across Salient360 and Sitzman training sets can fool existing metrics and achieve results on par with specialized models. Thus, we propose a new probabilistic metric based on the independent Bernoullis assumption that is more suited to the 360° saliency task.