Modeling focus of attention for meeting indexing

Abstract

Visual cues, such as gesturing, looking at each other or monitoring each others facial expressions, play an important role in meetings. Such information can be used for indexing of multimedia meeting recordings. In this paper, we present an approach to detect who is looking at whom during a meeting. Our proposal is to employ Hidden Markov Models to characterize participants’ focus of attention by using gaze information as well as knowledge about the number and positions of people present in a meeting. The number and positions of the participants faces are detected in the field of view of a panoramic camera. We use neural networks to estimate the directions of participants’ gaze from camera images. We discuss the implementation of the approach in detail including system architecture, data collection, and evaluation. The system has achieved an accuracy rate of up to 93 % in detecting focus of attention on test sequences taken from meetings. We have used focus of attention as an index in a multimedia meeting browser.