Head and hands 3D tracking in real time by the EM algorithm

This paper presents a method for real time hand and head tracking, in three dimensions, using two cameras. This tracking is intended as a first step for a gesture recognition system, using the trajectories of the hands, or as input to a real time clone animation system. The method used is based on simple preprocessing followed by the use of a statistical model linking the observations to the parameters: the position of the hands and the head. Preprocessing consist of background subtraction followed by skin color detection, using a simple color lookup table. The statistical model is composed of three ellipsoids, one for each hand and one for the head. A Gaussian probability density with the same center and size is associated with each ellipse. The parameters of the model are adapted to the pixel detected by the preprocessing stage. The EM algorithm is used to obtain the parameters corresponding to the maximum of the likelihood. The hands and head tracking is realized in near real time on a single workstation.