Gestures vs. Gesticulations: Change Point Models Based Segmentation for Natural Interactions

Abstract

Using gestures for natural interactions in virtual environments require robust and smart recognition systems. In these contexts, gestures and gesticulations are part of the a continuous information stream: the first are sufficient to convey meaningful information such as commands and indications. On the contrary, gesticulations are unconscious body movements performed mainly, to support speech. In the majority of gestures recognition systems, the implicit assumption of "isolated patterns" is made. Indeed, following the Kendon's morpho-kinetics model, a gesture is the part of the armmovement contained between the pre-stroke and the post-stroke. This strong assumption shifts the recognition problem toward a clustering issue, e.g., recognizing an isolated temporal pattern. From the practical point of view, the isolated gestures hypothesis needs a cooperation from the user and the later should emphasize the pre and the post strokes. This removes the naturalness of the targeted interface. In this contribution, we focus on having a strong segmentation technique that clusters the body movements into consistent sequences. In this paper, we present a non-parametric stochastic segmentation algorithm that is able to cluster the continuous time series representing body movements into gestures and non-gestures segments. We show as well how this technique allows any novice user creating in a semi-supervised way, his or her, own gestures library. The proposed system is assessed through a real-life example, where a novice user creates an adhoc interface to control an artificial agent in a natural way.