Content-based analysis for accessing audiovisual archives: Alternatives for concept-based indexing and search

Huge amounts of audiovisual material have been digitized recently, resulting in a great source of information relevant both from a cultural and historical point of view. However, in spite of millions of man hours spent on manual annotation and recent advances in (semi-)automatic metadata generation, accessing these archives and retrieving relevant information from them remains a difficult task. Up to recently, the main paradigm to open up archives by automatic tools for audiovisual analysis has been a concept-based indexing and retrieval oriented approach. However, this approach has its limitations, in that it does not scale well, it requires strong supervision, and does not really match well to the user's needs. In this paper, we discuss some upcoming alternative approaches that try to overcome or circumvent some of these issues. This includes i) the use of knowledge modeling to bridge the semantic gap; ii) on-the-fly learning of new, user-defined concepts; and iii) weakly supervised methods that learn from associated text data. We also discuss what we consider important open issues at this time that deserve more attention from the research community.