Enhancing the Perceived Emotional Intelligence of Conversational Agents through Acoustic Cues

Abstract

The perceived emotional intelligence of a conversational agent (CA) can significantly impact people’s interaction with the CA. Prior research applies text-based sentiment analysis and emotional response generation to improve CAs’ emotional intelligence. However, acoustic features in speech containing rich contexts are underexploited. In this work, we designed and implemented an emotionally aware CA, called HUE (Heard yoUr Emotion) that stylized responses with emotion regulation strategies and empathetic interjections. We conducted a user study with 75 participants to evaluate their perceived emotional intelligence (PEI) of HUE by having them observe conversations between people and HUE in different emotional scenarios. Our results show that participants’ PEI was significantly higher with the acoustic features than without.