Modelling conversational agent with empathy mechanism

Empathy mechanism in communication is the cornerstone for effective and meaningful interaction. Establishing an empathy mechanism in conversation agent (CA) requires accurate recognition of users’ emotions to facilitate generates appropriate empathetic responses. Therefore, we proposed a Multimodal Emotion Recognition Model (MERM) to recognizes a user’s emotional state from multimodal data (audio, facial expressions, and conversation text) during conversation, and an Interactive Empathetic Conversation Model (IECM) to generate empathetic responses based on the MERM. Comparative and ablation study results indicated that the proposed models outperform existing methods in recognizing the user’s emotions and generating appropriate empathetic responses. We also conducted an experiment study, the results indicated that the CA significantly enhances the user’s emotional experience.