Fine Tuning BERT for Unethical Behavior Classification

Abstract

Social media allows people to express themselves, however, there exists a threat of abuse and harassment. This threat leads to a negative impact on society which results in a change in people behaviour and they stop expressing their ideas freely. Classification of unethical behaviour in comments is a multi-label classification task. Due to the limited availability of the dataset, training does not yield worthy accuracies. Hence, a large training corpus is needed. This work, therefore, proposes to supplement training data by making use of transfer learning. Bi-directional Encoder Representations from Transformers (BERT) pre-trained model is fine-tuned to detect unethical users’ behaviour. The approach used in this work achieved competitive accuracy for the task of multi-label classification on the toxicity dataset of Wikipedia Comments Corpus.