COVID-19 Fake News Detection With Pre-trained Transformer Models

Abstract

COVID-19 is a new virus that first appeared in the year 2020 and is still currently plaguing our world. With the emergence of this virus, much information, both fake and real, has circulated in the internet. Fake information can lead to misleading information and cause a riot in society. In this paper, we aim to build a hoax detection system using the pre-trained transformer models BERT, RoBERTa, DeBERTa and Electra. From these four models, we will find which model gives the most accurate results. BERT gives a validation accuracy of 97.15% and test accuracy of 97.01%. RoBERTa gives a validation accuracy of 97.34% and test accuracy of 97.15%. DeBERTa gives a test accuracy of 97.48% and a test accuracy of 97.25%. Lastly, Electra gives a validation accuracy of 97.95% and a test accuracy of 97.76%. Electra is one of the newer models and is proven to be the most accurate model in our experiment and the one we will choose to implement fake news detection.