Head detection and tracking by 2-D and 3-D ellipsoid fitting

A novel procedure for segmenting a set of scattered 3D data obtained from a head and shoulders multiview sequence is presented. The procedure consists of two steps. In the first step, two ellipses corresponding to the head and the body of the person are identified based on ellipse fitting of the outline of the person in each image. The fitting is based on a fast direct least squares method using the constraint that forces a general conic to be an ellipse. In order to achieve head/body segmentation, a K-means algorithm is used to minimise the fitting error between the points and the two ellipsoids. In the second step, a 3D ellipsoid model corresponding to the head of the person is identified using an extension of the above method. Robustness and outlier removal can be achieved if a 3D ellipsoid model estimation technique is used in conjunction with the Median of Least Squares (MedLS) technique, which minimises the median of the errors corresponding to each 3D point. An interesting application of the proposed method is the combination of the 3D ellipsoid model with a generic face model which is adapted to the face images to provide information only for the high-detail front part of the head while the 3D ellipsoid is used for the back of the head, which is usually not visible.