Game Character Facial Animation Using Actor Video Corpus and Recurrent Neural Networks

Abstract

Creating photorealistic facial animation for game characters is a labor-intensive process that gives authorial primacy to animators. This research presents an experimental autonomous animation controller based on an emotion model that uses a team of embedded recurrent neural networks (RNNs). The design is a novel alternative method that can elevate an actor’s contribution to game character design. This research presents the first results of combining a facial emotion neural network model with a workflow that incorporates actor preparation methods and the training of auto-regressive bi-directional RNNs with long short-term memory (LSTM) cells. The predicted emotion vectors triggered by player facial stimuli strongly resemble a performing actor for a game character with accuracies over 80% for targeted emotion labels and show accuracy near or above a high baseline standard.