Action Unit Generation through Dimensional Emotion Recognition from Text

Abstract

Expressiveness is a critical feature for the communication between humans and robots, and it helps humans to better understand and accept a robot. Emotions can be expressed through a variety of modalities: kinesthetic (via facial expression), body posture and gestures, auditory, thus the acoustic features of speech, and semantic, thus the content of what is said. One of the most effective modalities to communicate emotions is through facial expressions. Social robots often show facial expressions with coded animations. However, the robot must be able to express appropriate emotional responses according to the interaction with people. In this work, we consider verbal interactions between humans and robots and propose a system composed of two modules for the generation of facial emotions by recognising the arousal and valence values of a written sentence. The first module, based on Bidirectional Encoder Representations from Transformers, is deployed for emotion recognition in a sentence. The second, an Auxiliary Classifier Generative Adversarial Network, is proposed for the generation of facial movements for expressing the recognised emotion in terms of valence and arousal.