Analyzing common lexical features of fake news using multi-head attention weights

Abstract

Numerous approaches have been developed to identify fake news through machine learning; however, these methods are predominantly assessed using singular datasets specific to certain fields, leading to a scarcity of research on versatile models adaptable to a range of domains. This study evaluates the adaptability of a fake news detection model across diverse fields, employing three distinct datasets. Furthermore, the study leverages the multi-head attention feature of bidirectional encoder representations from transformers (BERT) to scrutinize the feature extraction process in the model. In our analysis, we focused on words that are commonly emphasized by machine learning in fake news detection. The dataset comprised 27,442 instances of genuine news and 28,359 instances of fabricated news, each distinctly labeled. To examine the focal words, we utilized multi-head attention, a component of BERT. This mechanism assigns greater weight to words that receive more attention. Our investigation aimed to identify which words were assigned higher weights in each article. The findings indicate that while representing a minor percentage, a common characteristic of fake news is the heightened attention to words that influence the credibility of the article. To assess the versatility of the model, we applied the model trained on one dataset to classify other datasets. The results demonstrate a notable decline in accuracy, attributable to the distinctive characteristics of the training data. These observations suggest that common features among fake news, which could be extracted using the fine-tuned BERT model, are limited.