Silver Lining in the Fake News Cloud: Can Large Language Models Help Detect Misinformation?

Abstract

In the times of advanced generative artificial intelligence, distinguishing truth from fallacy and deception has become a critical societal challenge. This research attempts to analyze the capabilities of large language models (LLMs) for detecting misinformation. Our study employs a versatile approach, covering multiple LLMs with few- and zero-shot prompting. These models are rigorously evaluated across various fake news and rumor detection datasets. Introducing a novel dimension, we additionally incorporate sentiment and emotion annotations to understand the emotional influence on misinformation detection using LLMs. Moreover, to extend our inquiry, we employ ChatGPT to intentionally distort authentic news as well as human-written fake news, utilizing zero-shot and iterative prompts. This deliberate corruption allows for a detailed examination of various parameters such as abstractness, concreteness, and named entity density, providing insights into differentiating between unaltered news, human-written fake news, and its LLM-corrupted counterpart. Our findings aspire to furnish a refined framework for discerning authentic news, human-generated misinformation, and LLM-induced distortions. This multifaceted approach, utilizing various prompt techniques, contributes to a comprehensive understanding of the subtle variations shaping misinformation sources.