Enhancing text-centric fake news detection via external knowledge distillation from LLMs

Abstract

Fake news poses a significant threat to society, making the automatic and accurate detection of fake news an urgent task. Various detection cues have been explored in extensive research, with news text content shown to be indispensable as it directly reflects the creator’s intent. Existing paradigms for developing text-centric methods, i.e., small language model (SLM)-based, external knowledge-enhanced, and large language model (LLM)-based approaches, have achieved remarkable improvements. However, each of these paradigms still faces the following challenges: (1) the low generalization ability of SLM-based methods, due to their training on limited and specific knowledge; (2) the extensive retrieval operations required by external knowledge-enhanced methods, both during training and at the inference stage, leading to increased computational costs; and (3) LLMs are prone to hallucinations and less suited for factual reasoning. To address these challenges, we propose LEKD, which combines the strengths of SLMs, external knowledge, and LLMs to enhance text-centric fake news detection. Specifically, LEKD leverages the LLM to generate external knowledge as supplementary information for the training set only and introduces a graph-based semantic-aware feature alignment module to resolve knowledge contradictions, as well as an information bottleneck-based knowledge distillation module to ensure the implicit generation of these features during inference. Extensive experiments conducted on two datasets demonstrate the advantages of LEKD over the baselines.