Transformer Models for Recognizing Abusive Language An investigation and review on Tweeteval and SOLID dataset

Abstract

Social engineering communities have become very popular among the kids and elderly alike. In this era of social media, the streaming of comments, opinions, reviews and communications is done via most common social media messaging communities like Twitter, Meta owned WhatsApp, FB and Instagram, Snapchat, telegram and YouTube comments. In this paper we perform a review on the different methods and models used to identify the offensive language using different datasets. Offensive language detection is a tedious task as it is country and language specific. The corpus used to identify the offensiveness and abusiveness is not covering all the word usages. We have done a comparison study of different methods on text to detect the post is offensive or not. The detection of abusive language is an unsolved and challenging problem to researchers in Natural Language Processing (NLP). This has led to be one of the reasons for increased level of mental instability among teenagers to elderly. The crime via social media has increased to a large value than older days. The study and surveys show that to recognize the structure and context of the language is the best way to solve this problem to an extent. The paper aims to four recent transformer models pretrained and fine-tuned for offensive language detection on the tweeteval dataset viz; DistilBERT, RoBERTa, DistilRoBERTa and DeBERTa. All the model had limitation in the performance based on the training data size used but are optimized by tuning hyper parameters during training. The models are limited to English language offensive words and recent works are going on in the area of multilingual tweets on both text and speech processing.