Exploiting visual reranking to improve pseudo-relevance feedback for spoken-content-based video retrieval

In this paper we propose an approach that utilizes visual features and conventional text-based pseudo-relevance feedback (PRF) to improve the results of semantic-theme-based video retrieval. Our visual reranking method is based on an Average Item Distance (AID) score. AID-based visual reranking is designed to improve the suitability of items at the top of the initial results list, i.e., those feedback items selected for use in query expansion. Our method is intended to help target feedback items representative of visual regularity typifying the semantic theme of the query. Experiments performed on the VideoCLEF 2008 data set and on a number of retrieval scenarios combining the inputs from speech-transcript-based (i.e., text-based) search and visual reranking demonstrate the benefits of using AID-based visual representatives to compensate for the inherent problems of PRF, such as topic drift.