Video saliency based on rarity prediction: Hyperaptor

Abstract

Saliency models are able to provide heatmaps highlighting areas in images which attract human gaze. Most of them are designed for still images but an increasing trend goes towards an extension to videos by adding dynamic features to the models. Nevertheless, only few are specifically designed to manage the temporal aspect. We propose a new model which quantifies the rarity natively in a spatiotemporal way. Based on a sliding temporal window, static and dynamic features are summarized by a time evolving "surface" of different features statistics, that we call the "hyperhistogram". The rarity-maps obtained for each feature are combined with the result of a superpixel algorithm to have a more object-based orientation. The proposed model, Hyperaptor stands for hyperhistogram-based rarity prediction. The model is evaluated on a dataset of 12 videos with 2 different references along 3 different metrics. It is shown to achieve better performance compared to state-of-the-art models.