Annotation of still images by multiple visual concepts

Abstract

The automatic indexing of images and videos is a highly relevant and important research area in the field of multimedia information retrieval. The difficulty of this task is no longer something to prove. The majority of the efforts of the research community have been focused in the past on the detection of single concepts in images/videos, which is already a hard task. With the evolution of the information retrieval systems, users needs are more abstract, and lead to a larger number of words composing the queries. It is sensible to think about indexing multimedia documents by more than one concept, to help retrieval systems to answer such complex queries. Few studies addressed specifically the problem of detecting multiple concepts (multi-concept) in images and videos, most of them concern the detection of concept pairs. These studies showed that such challenge is even greater than the one of single concept detection. In this work, we address this problematic of mult-concept detection in still images. Two types of approaches are considered : 1) building models per multi-concept and 2) fusion of single concepts detectors. We conducted our evaluation on PASCAL VOC'12 collection regarding the detection of pairs and triplets of concepts. Our results show that the two types of approaches give globally comparable results, but they differ for specific kinds of pairs/triplets.