A virtual reality agent-based platform for improvisation between real and virtual actors using gestures

Abstract

This paper describes the process of designing a Virtual Reality Agent-based platform to explore an interactive gestural dialogue between real and virtual actors and propose an analysis of the results. It is part of the CIGALE interdisciplinary project involving researchers in the fields of linguistics, digital arts and computer science, as well as actors and professors. Several experiments of improvisation conducted with actors through computer-mediated interactions help in characterizing specific dynamics of gestures, recurrent behaviors and gestural patterns. We extract specific descriptors of movement and behaviors that will drive the improvisation. The virtual actor is able to improvise and generate its gestures using a finite state machine that triggers three main behaviors according to the actor's body posture and arm movements: the virtual actor can mime in real-time the real actor, it can propose expressive or linguistic gestures from four MoCap databases, or it can generate a new gesture using a genetic algorithm. A first installation experienced by the participants during an international art festival, shows they are able to improvise fluently with the virtual actor in an expressive dialogue.