Particle filtering for bearing-only audio-visual speaker detection and tracking

We present a method for audio-visual speaker detection and tracking in a smart meeting room environment based on bearing measurements and particle filtering. Bearing measurements are determined using the Time Difference of Arrival (TDOA) of the acoustic signal reaching a pair of microphones, and by tracking facial regions in images from monocular cameras. A particle filter is used to sample the space of possible speaker locations within the meeting room, and to fuse the bearing measurements from auditory and visual sources. The proposed system was tested in a video messaging scenario, using a single participant seated in front of a screen to which a camera and microphone pair are attached. The experimental results show that the accuracy of speaker tracking using bearing measurements is related to the location of the speaker relative to the locations of the camera and microphones, which can be quantified using a parameter known as Dilution of Precision.