Content-based image retrieval by multimodal interaction

Abstract

Due to the size of today's professional image databases, the standard approach to content-based image retrieval is to interactively navigate through the content. However, most people whose job necessitates working with such databases do not have a technical background. Commercial practice thus requires efficient retrieval techniques as well as navigation interfaces that are intuitive to use and easy to learn. In this paper we introduce a system for interactive image retrieval that combines different approaches to feature based queries. Furthermore, it allows multimodal interaction because apart from conventional input devices like mouse and keyboard, it is possible to operate the system using a touch screen or even natural language. Besides technical details and results on retrieval accuracy, we also present results of usability experiments which underline that users well appreciate multimodal interfaces for image retrieval.