ProperBERT - Proactive Recognition of Offensive Phrasing for Effective Regulation

Abstract

This work discusses and contains content that may be offensive or unsettling. Hateful communication has always been part of human interaction, even before the advent of social media. Nowadays, offensive content is spreading faster and wider through digital communication channels. To help improve regulation of hate speech, we introduce ProperBERT, a fine-tuned BERT model for hate speech and offensive language detection specific to English. To ensure the portability of our model, five data sets from literature were combined to train ProperBERT. The pooled dataset contains racist, homophobic, misogynistic and generally offensive statements. Due to the variety of statements, which differ mainly in the target the hate is aimed at and the obviousness of the hate, a sufficiently robust model was trained. ProperBERT shows stability on data sets that have not been used for training, while remaining efficiently usable due to its compact size. By performing portability tests on data sets not used for fine-tuning, it is shown that fine-tuning on large scale and varied data leads to increased model portability.