Identifying offensive memes in low-resource languages: A multi-modal multi-task approach using valence and arousal

Abstract

Social media platforms, including Facebook, Twitter, and Instagram, have provided a revolutionary communication platform with unrestricted expression. However, this has also led to the propagation of offensive and abusive content, cyberbullying, and harassment. The use of memes, a popular form of multimodal media, has grown exponentially and is often used to spread objectionable content through the use of dark humor. In this paper, we propose a multi-task multi-modal framework for identifying offensive Hindi memes by leveraging the auxiliary tasks of valence and arousal to improve model performance. This approach leads to a more nuanced understanding of offensive memes and outperforms unimodal models that consider only one modality. To facilitate future research, we present a new Hindi corpus, named OffVA, containing 7,646 Hindi memes annotated with offensive, valence, and arousal labels. This is the first dataset of its kind for Hindi and can serve as a benchmark for future research on detecting offensive content in Hindi memes. Additionally, we emphasize the importance of incorporating high-resource language datasets, such as English, in identifying offensive memes in low-resource languages to improve model performance. Our experimental results on this dataset demonstrate that the proposed framework outperforms unimodal models in identifying offensive memes, and the incorporation of valence and arousal as auxiliary tasks leads to better results, highlighting the importance of considering multiple modalities and tasks for effective offensiveness detection in memes.