Detection and Estimation of Omni-Directional Pointing Gestures Using Multiple Cameras

Abstract

We propose a multi-camera system that can detect omni-directional pointing gestures and estimate the direction of pointing. In general, when a human points at something, their target exists directly in front of the direction they are facing. Therefore, we regard the direction of pointing as the direction represented by the straight line that connects the face position with the hand position. First, the multiple cameras detect the face region by skin colors and estimate the face direction with the discrete face direction feature classes. Second, we estimate the precise direction that the subject is facing with the integrated information from multiple cameras and decide which camera captures the frontal view of the face the best. This camera is labeled the center camera. Third, we select a pair of cameras on both sides of the center camera as a stereo camera and detect the spatial positions of the face and hand. Finally, the target that the subject is pointing to is found on the straight line that connects the face position with the hand position. Experiments show that out system can achieve a mean error of 1.94" with a variance of 4.37 throughout the pointing direction.