A sarcasm detection method based on modality inconsistencies and textual knowledge enhancement

Abstract

Sarcasm detection aims to identify emotional tendencies in tweets, which helps governments and enterprises monitor online public opinions. The Twitter platform can create messages, including images and texts. Existing sarcasm detection methods mainly focus on extracting high-level semantic information from images while ignoring textual information. However, previous research has demonstrated that text is more important than images in sentiment analysis tasks. Inspired by this, we reduce the involvement of image information and investigate the sarcasm detection from a textual perspective. First, we divide the text in the primary dataset into pure text and hashtags. The hashtags are fused with high-frequency words in the pure text. Then, considering the differences on the data distribution between the training corpus of Bidirectional Encoder Representation from Transformers (BERT) and the sarcasm detection corpus, we use the Twitter sentiment analysis corpus to further pre-train the BERT model, obtaining the Basic_BERT and Hash_BERT models as feature extractors for the pure text and hashtags. Furthermore, to better play the role of the text in this task, a cross-gate mechanism method is proposed by a cross-attention transformer module and a similarity constraint. The cross-attention transformer module is used to generate a representation of intra-modal and inter-modal fusion while the similarity constraint is used to achieve a balance between the original modal representation and the fused modal representation. On the sarcasm detection dataset, the proposed model achieves an F1-score of 87.22%, an improvement of 3.30% over the most advanced model.