Semantically-aware blendshape rigs from facial performance measurements

Abstract

We present a framework for automatically generating personalized blendshapes from actor performance measurements, while preserving the semantics of a template facial animation rig. Firstly, we capture various poses from the subject with our photogrammetry apparatus. The 3D reconstruction from each pose is then corresponded by an image-based tracking algorithm. The core of our framework is an optimization algorithm which iteratively refines the initial estimation of the blendshapes such that they can fit the performance measurements better. This framework facilitates creation of an ensemble of realistic digital-double face rigs for each individual with consistent behavior across the character set.