Attention-based Model for Multi-modal sentiment recognition using Text-Image Pairs

Abstract

Multi-modal sentiment recognition (MSR) is an emerging classification task that aims to categorize sentiment polarities for a given multi-modal dataset. The majority of work done in the past relied heavily on text-based information. However, in many scenarios, text alone is frequently insufficient to predict sentiment accurately; as a result, academics are more motivated to engage in the subject of MSR. In light of this, we proposed an attention-based model for MSR using image-text pairs of tweets. To effectively capture the vital information from both modalities, our approach combines BERT and ConvNet with CBAM (convolution block attention module) attention. The outcomes of our experimentations on the Twitter-17 dataset demonstrate that our method is capable of sentiment classification accuracy that is superior to that of competing approaches.