Saliency-aware color moments features for image categorization and retrieval

Abstract

Traditional window-based color indexing techniques have been widely used in image analysis and retrieval systems. In the existing approaches, all the image regions are treated with equal importance. However, some image areas carry more information about their content (e.g. the scene foreground). The human visual system bases indeed the categorization process on such set of perceptually salient region. Therefore, in order to improve the discriminative abilities of the color features for image recognition, higher importance should be given to the chromatic characteristics of more informative windows. In this paper, we present an informativeness-aware color descriptor based on the Color Moments feature [17]. We first define a saliency-based measure to quantify the amount of information carried by each image window; we then change the window-based CM feature according to the computed local informativeness. Finally, we show that this new hybrid feature outperforms the traditional Color Moments in a variety of challenging dataset for scene categorization, object recognition and video retrieval.