Fighting Fake News Using Deep Learning: Pre-trained Word Embeddings and the Embedding Layer Investigated

Abstract

Fake news is progressively becoming a threat to individuals, society, news systems, governments and democracy. The need to fight it is rising accompanied by various researches that showed promising results. Deep learning methods and word embeddings contributed a lot in devising detection mechanisms. However, lack of sufficient datasets and the question “which word embedding best captures content features” have posed challenges to make detection methods adequately accurate. In this work, we prepared a dataset from a scrape of 13 years of continuous data that we believe will narrow the gap. We also proposed a deep learning model for early detection of fake news using convolutional neural networks and long short-term memory networks. We evaluated three pre-trained word embeddings in the context of the fake news problem using different measures. Series of experiments were made on three real world datasets, including ours, using the proposed model. Results showed that the choice of pre-trained embeddings can be arbitrary. However, embeddings purely trained from the fake news dataset and pre-trained embeddings allowed to update showed relatively better performance over static embeddings. High dimensional embeddings showed better results than low dimensional embeddings and this persisted for all the datasets used.