MIAN: Multi-head Incongruity Aware Attention Network with transfer learning for sarcasm detection

Abstract

Sarcasm is a common rhetorical metaphor in social media platforms, that individuals express emotion contrary to the literal meaning. Capturing the incongruity in the texts is the critical factor in sarcasm detection. Although several studies have looked at the incongruity of a single text, there is currently a lack of studies on modeling the incongruity of contextual information. Inspired by Multi-Head Attention mechanism from Transformer, we propose a Multi-head Incongruity Aware Attention Network, which concentrates on both target semantic incongruity and contextual semantic incongruity. Specifically, we design a multi-head self-match network to capture target semantic incongruity in a single text. Moreover, a multi-head co-match network is applied to model the contextual semantic incongruity. Furthermore, due to the scarcity of sarcasm data and considering the correlation between tasks of sentiment analysis and sarcasm detection, we pre-train the language model with a great amount of sentiment analysis data, which enhances its ability to capture sentimental features in the text. The experimental results demonstrate that our model achieves state-of-the-art performance on four benchmark datasets, with an accuracy gain of 3.8% on Tweets Ghost, 1.1% on SARC Pol, 1.9% on Ciron and an F1-Score gain of 0.3% on FigLang Twitter.