SmartRAN: Smart Routing Attention Network for multimodal sentiment analysis

Abstract

Multimodal sentiment analysis has received widespread attention from the research community in recent years; it aims to use information from different modalities to predict sentiment polarity. However, the model architecture of most existing methods is fixed, and data can only flow along an established path, which leads to poor generalization of the model to different types of data. Furthermore, most methods explore only intra- or intermodal interactions and do not combine the two. In this paper, we propose the Smart Routing Attention Network (SmartRAN). SmartRAN can smartly select the data flow path on the basis of the smart routing attention module, effectively avoiding the disadvantages of poor adaptability and generalizability caused by a fixed model architecture. In addition, SmartRAN includes the learning process of both intra- and intermodal information, which can enhance the semantic consistency of comprehensive information and improve the learning ability of the model for complex relationships. Extensive experiments on two benchmark datasets, CMU-MOSI and CMU-MOSEI, prove that the proposed SmartRAN has superior performance to state-of-the-art models.