Emotion Recognition In Emergency Call Centers: The challenge of real-life emotions

Abstract

Detected emotional states of speakers are a key component of constructive social relationships but also of efficiency for capturing the degree of emergency. This paper provides an overview of my doctoral project that focuses on bimodal emotion recognition in an emergency call center with deep end-to-end learning techniques using the most advanced approaches such as transformer and zero-shot learning. In this work, we will first propose a supervised classification system for bimodal emotion recognition (paralinguistic and linguistic). Then, we will investigate an unsupervised system as a complement to the previous one in order to deal with “unseen” emotions and mixtures of real-life emotions. Our previous studies mainly explored the acoustic modality of speech emotion recognition (SER), we achieved close to the state-of-the-art results on the improvised part of the well-known database IEMOCAP and we applied our approach to a French emergency database CEMO collected in a previous project. In my thesis, new real recordings in an emergency call center will be collected. The main research topics of my thesis are: Emotional representation and annotation; Speech emotion recognition and ethical implications; Evaluation and real-life trials.