Is it fake or not? A comprehensive approach for multimodal fake news detection

Abstract

In recent years, the proliferation of fake news has posed significant challenges to information integrity and public trust, paving the way for the development of artificial intelligence-based models that can analyze information and determine its veracity. This study comprehensively evaluates the Themis architecture in the context of fake news detection on two distinct public datasets: Fakeddit and ReCoVery. To enhance model performance, we systematically investigate various customizations of Themis, including the integration of Low-Rank Adaptation, diverse data augmentation techniques, and multiple configurations, employing the TinyLlama Large Language Model and CLIP ViT image encoders while tuning key parameters to optimize results. Our findings reveal that while the standard Themis model performed adequately, significant improvements were observed by incorporating LoRA and specific data augmentation strategies, particularly in the ReCoVery dataset. Comparisons with existing literature indicate that Themis achieves competitive performance, especially in the ReCoVery dataset, where it outperforms existing solutions.