An Empirical Comparison of BERT, RoBERTa, and Electra for Fact Verification

Abstract

We reviewed some features of a number of fact verification techniques by comparing 3 (three) algorithms, namely BERT, RoBERTa, and Electra. These 3 (three) algorithms have different advantages, i.e., BERT and RoBERTa predict hidden words using a huge dataset, and Electra verifies facts by detecting tokens that are replaced in a text or sentence. It is necessary to find the model with a good performance evaluation value to produce the best fact verification results. The evaluation of the performance model in this study uses the F1-Score. Our experimental results show that RoBERTa achieves the best accuracy and F1-Score with a value of 95.4% and 95.3% with the parameter value of epoch of 5 (five) and a batch size of 32. The second position is occupied by BERT, with the best result of accuracy and F1-Score at the same value of 94.3% with the epoch of10 (ten) and a batch size of32. Although it provides a shorter elapsed time, unfortunately, Electra does not outperform other models in fact verification.