Enhanced Feature Representation for Multimodal Fake News Detection Using Localized Fine-Tuning of Improved BERT and VGG-19 Models

Abstract

The spread of fake news poses significant challenges across various sectors, including health, the economy, politics, and national stability. Social media and modern technology have facilitated the rapid dissemination of fake news, predominantly in multimedia formats. Despite advancements, multimodal fake news detection models struggle to achieve optimal accuracy, primarily due to the quality of feature representation. This study aims to enhance feature representation to improve fake news identification. Pre-trained models for feature extraction, typically designed for general public domains, may not suit the specific characteristics of our task using the Fakeddit dataset. We propose a localized fine-tuning strategy, refining pre-trained BERT and VGG-19 models for accurate multimodal feature representation in fake news detection. BERT was fine-tuned by retraining all layers, while only the last block of VGG-19 was fine-tuned. To further enhance the representations, we made structural modifications to VGG-19, including the use of a global average pooling layer and a redesigned classifier. This approach significantly improved our multimodal fake news detection model’s performance, achieving a high accuracy of 92%. Compared to state-of-the-art studies that use generic pre-trained models, our model demonstrates superior performance. Our research underscores the importance of feature representation in multimodal contexts and opens avenues for exploring the synergy between textual and visual modalities in fake news detection.