Continuous control of avatar gesture

Abstract

We are developing an application to give humans the ability to transmit nonverbal communication behaviors through an avatar: specifically gesture, the movements of the arms and hands that accompany speech when people speak face-to-face. In this application the user will have continuous control over the avatar animation. The avatar will be like a virtual puppet and the user will manipulate the avatar using not strings or rods but the controlled and skilled motions of their hand. The system tracks hand motions and then maps that motion to the joint motions of a three-dimensional articulated avatar. As part of this research we will try out different ways of tracking the user's hand. Eventually we plan to test the efficacy of this system by incorporating it into a networked virtual environment in which two or more people can interact through the virtual medium. Working with artists will enable us to design a system that is expressive and to better understand the expressive power of this system.