Facial retargeting using neural networks

Abstract

Mapping the motion of an actor's face to a virtual model is a difficult but important problem, especially as fully animated characters are becoming more common in games and movies. Many methods have been proposed but most require the source and target to be structurally similar. Optical motion capture markers and blendshape weights are an example of topologically incongruous source and target examples that do not have a simple mapping between one another. In this paper, we created a system capable of determining this mapping through supervised learning of a small training dataset. Radial Basis Function Networks (RBFNs) have been used for retargeting markers to blendshape weights before but to our knowledge Multi-Layer Perceptron Artificial Neural Networks (referred to as ANNs) have not been employed in this way. We hypothesized that ANNs would result in a superior retargeting solution compared to the RBFN, due to their theoretically greater representational power. We implemented a retargeting system using ANNs and RBFNs for comparison. Our results found that both systems produced similar results (figure 1) and in some cases the ANN proved to be more expressive although the ANN was more difficult to work with.