A Multi-View Co-Learning Method for Multimodal Sentiment Analysis

Abstract

Existing works on multimodal sentiment analysis have focused on learning more discriminative unimodal sentiment information or improving multimodal fusion methods to enhance modal complementarity. However, practical results of these methods have been limited owing to the problems of insufficient intra-modal representation and inter-modal noise. To alleviate this problem, we propose a multi-view co-learning method (MVATF) for video sentiment analysis. First, we propose a multi-view features extraction module to capture more perspectives from a single modality. Second, we propose a two-level fusion sentiment enhancement strategy that uses hierarchical attentive learning fusion and a multi-task learning fusion module to achieve co-learning to effectively filter inter-modal noise for better multimodal sentiment fusion features. Experimental results on the CH-SIMS, CMU-MOSI and MOSEI datasets show that the proposed method outperforms the state-of-the-art methods.