Saliency map driven image retrieval combining the bag-of-words model and PLSA

Abstract

A new image retrieval system is proposed that combines the bag-of-words (BoW) model and Probabilistic Latent Semantic Analysis (PLSA). First, interest points on images are detected using the Hessian-Affine keypoint detector and Scale Invariant Feature Transform (SIFT) descriptors are computed. Graph-based visual saliency maps are then employed in order to detect and discard outliers in image descriptors. By doing so, SIFT features lying in non-salient regions can be deleted. All the remaining reliable feature descriptors are divided into a number of subsets and partial vocabularies are extracted for each of them. The final vocabulary used in the BoW model is obtained by the concatenating the partial vocabularies. The resulting BoW representations are weighted using the TF-IDF scheme. Finally, the PLSA is employed to perform a probabilistic mixture decomposition of the weighted BoW representations. Query expansion is demonstrated to improve the retrieval quality. Overall a 0.79 mean average precision is reported when the saliency filtering was applied on SIFTs and the BoW plus PLSA method was used.