Veracity-Oriented Context-Aware Large Language Models–Based Prompting Optimization for Fake News Detection

Abstract

Fake news detection (FND) is a critical task in natural language processing (NLP) focused on identifying and mitigating the spread of misinformation. Large language models (LLMs) have recently shown remarkable abilities in understanding semantics and performing logical inference. However, their tendency to generate hallucinations poses significant challenges in accurately detecting deceptive content, leading to suboptimal performance. In addition, existing FND methods often underutilize the extensive prior knowledge embedded within LLMs, resulting in less effective classification outcomes. To address these issues, we propose the CAPE–FND framework, context-aware prompt engineering, designed for enhancing FND tasks. This framework employs unique veracity-oriented context-aware constraints, background information, and analogical reasoning to mitigate LLM hallucinations and utilizes self-adaptive bootstrap prompting optimization to improve LLM predictions. It further refines initial LLM prompts through adaptive iterative optimization using a random search bootstrap algorithm, maximizing the efficacy of LLM prompting. Extensive zero-shot and few-shot experiments using GPT-3.5-turbo across multiple public datasets demonstrate the effectiveness and robustness of our CAPE–FND framework, even surpassing advanced GPT-4.0 and human performance in certain scenarios. To support further LLM–based FND, we have made our approach’s code publicly available on GitHub (our CAPE–FND code: https://github.com/albert-jin/CAPE-FND [Accessed on 2024.09]).