Implementation of Single Camera Markerless Facial Motion Capture using Blendshapes

Abstract

Facial motion capture is the process of digitizing the facial motion of an actor by locating several facial landmarks of the actor’s face and using the relative coordinates of these landmarks to drive the facial structure of a 3D character in software like Blender. Recent advances have enabled markerless technology to track the desired facial features from frame to frame. In this work, the input from a single front-facing camera is used and the face is located using face detection algorithms. Its output is then used to find the relative coordinates of facial landmarks like lip corners, upper eyelids, and eyebrows etc., using a facial landmark detector. To achieve comparable levels of accuracy without the depth or 3D information that would be captured from a multi-camera setup, morph targets have been used to add constraints to the animation to avoid unnatural positions of the virtual character. The distance between a referential landmark that has minimal movement and the driving landmark determines the influence of the corresponding morph target. To establish an orientation invariant landmark detection, geometric normalization and face size normalization have been deployed.