Speech-Driven Facial Animation by LSTM-RNN for Communication Use

Abstract

The goal of this research is developing a system that a rich facial animation can be used in communication is generated from only speech. A high-quality 3DCG character to be used in VR is quite popular. Also, the facial animation is important in enhancing the reality of the high-quality 3DCG character. Generally, a source of the generating facial animation is a camera. The problem is one of selecting optical cameras as an input. Using cameras as an input source, it causes limitations of the angle of view of the camera or problems that cannot be aware of the human face, depending on the orientation of the face. Therefore, it is reasonable for developing a system for generating a facial animation using only voice. In this study, we generate facial expressions from only speech using LSTM-RNN and showed the usefulness of the system in communication via user study.