Analysis and Modeling of Affective Audio Visual Speech Based on PAD Emotion Space

Abstract

This paper analyzes acoustic and visual features for affective audio-visual speech based on PAD (Pleasure-Arousal- Dominance) emotion space. The selected acoustic features include FO maximum, FO minimum, duration and energy. A set of Partial Expression Parameters (PEP) is proposed as visual features to describe affective facial movement on talking face. This paper explores the connection between PAD emotion space and acoustic/visual features respectively. The variation of acoustic features is predicted by PAD values, and a PAD-PEP mapping function for facial expression synthesis is built. Experimental result shows that PAD could be properly applied in describing emotional state as well as predicting the acoustic/visual features for affective audiovisual speech synthesis.