The Hand Mouse: GMM hand-color classification and mean shift tracking

Abstract

This paper describes an algorithm to detect and track a hand in each image taken by a wearable camera. We primarily use color information, however, instead of pre-defined skin-color models, we dynamically construct hand- and background-color models by using a Gaussian mixture model (GMM) to approximate the color histogram. Not only to obtain the estimated mean of hand color necessary for the restricted EM algorithm that estimates the GMM but also to classify hand pixels based on the Bayes decision theory, we use a spatial probability distribution of hand pixels. Since the static distribution is inadequate for the hand-tracking stage, we translate the distribution with the hand motion based on the mean shift algorithm. Using the proposed method, we implemented the Hand Mouse that uses the wearer's hand as a pointing device, on our wearable vision system.