TransIEA: Transformer-Baseartd Image Emotion Analysis

Abstract

The application of the so-called Transformer network for natural language sentiment recognition is well established. It contains a self-attentive mechanism that allows for better learning based on the context. This mechanism facilitates the analysis of emotional content. Similar to natural language sentiment analysis, image emotion analysis also needs to combine the context in the image, i.e., global and local features of the image to analyze must be combined. No previous studies validated the performance of Transformer in image emotion analysis. In this study, we applied a new approach. For the first time, we aimed at classifying emotion images on the basis of the Transformer network. A new module for convolutional-neural-network-based feature extraction was added in front of the network. The conducted experimental analysis show that our network model outperforms most of the deep-learning models on the commonly used emotion image classification dataset, i.e., the FI (Facebook and instagram) dataset. The model achieves a classification accuracy of 73.40% on this dataset.