ICANet: A Method of Short Video Emotion Recognition Driven by Multimodal Data

Abstract

With the fast development of artificial intelligence and short videos, emotion recognition has become one of the most important research topics in human-computer interaction. At present, most emotion recognition methods still stay in a single modality. However, in daily life, human beings will usually disguise their real emotions, which leads to the problem that the low accuracy of single modal emotion recognition. Moreover, it is not easy to distinguish similar emotions. Therefore, we propose a new approach denoted as ICANet to achieve multimodal short video emotion recognition by employing three different modalities of audio, video, and optical flow, making up for the lack of a single modality and then improving the accuracy of emotion recognition in short videos. ICANet has a better accuracy of 80.77% on the IEMOCAP benchmark. The cross-modal fusion method of short video emotion recognition established in this paper can effectively improve the accuracy of emotion recognition in human-computer interaction scenarios.