Realtime Performance-Based Facial Avatars for Immersive Gameplay

Abstract

This paper discusses how realtime face tracking and performance-based animation can facilitate new ways of interaction for computer gaming and other online applications. We identify a number of requirements for realtime face capture and animation systems, and show how recent progress in tracking algorithms advances towards satisfying these requirements. Performance-based animation has the potential to significantly enrich the emotional experience of in-game communication through live avatars that are controlled directly by the facial expressions of a recorded user. We briefly outline other potential use scenarios of realtime face tracking systems and conclude with a discussion of future research challenges.