Towards model-based gesture recognition

Abstract

We propose a new technique for gesture recognition that involves both physical and control models of gesture performance, and describe preliminary experiments done to validate the approach. The technique incorporates underlying dynamics and control models are used to augment a set of Kalman-filter-based recognizer modules so that each filters the input data under the a priori assumption that one of the gestures is being performed. The recognized gesture is the filter output that most closely matches the output of an unaugmented Kalman filter. In our preliminary experiments, we treated gestures made with simple motions of the right arm, done while tracking only hand position. We modeled the path that the hand traverses while performing a gesture as a point-mass moving through air. The control model for each specific gesture was simply an experimentally determined sequence of applied forces plus a proportional control based on spatial position. Our experiments showed that even using such a simple set of models we were able to obtain results reasonably comparable with a carefully hand-constructed feature-based discriminator on a limited set of spatially-distinct planar gestures.