Sentiment Analysis of Persian Instagram Post: a Multimodal Deep Learning Approach

Abstract

Instagram is a popular social media that has a wide range of active users from ordinary people to artists and official users. Instagram posts are widely used by users to share text, image or video. Many users use text to describe or complement the images they share. To analyze the sentiment of such posts, both the content of the text and the image should be considered at the same time. This requires modelling of the relationship between the text and image modalities. To address this problem, we propose a multimodal deep learning method. The proposed method utilizes a bi-directional gated recurrent unit (bi-GRU) for processing text comments and a 2-dimensional convolutional neural network (2CNN) for analyzing images. In order to assess the performance of the proposed model, we introduce a new dataset of Instagram posts, MPerInst, containing 512 pairs of images and their corresponding comments written in the Persian language. Implementation results shows that employing both text and image modalities improves polarity detection accuracy and F1-scrore by 23% and 0.24 compared to using only image and text modalities, respectively. Moreover, the proposed model outperforms 11 similar deep fusion models by 11% and 0.1 in terms of accuracy and F1-score. Both the dataset and the codes of our proposed model are publicly available for probable future use.