Multi-granularity alignment and cross-modal reasoning for fake news video explanation

Abstract

Fake news video explanation generation aims to provide accurate and insightful explanations through in-depth analysis of news video content. However, existing methods typically align video context with overall descriptions and generate explanations via multi-modal fusion, often neglecting the rich details of key semantic elements such as nouns and verbs. To address this limitation, this paper proposes a unified Artificial Intelligence (AI) framework named Multi-Granularity Alignment and Reasoning (MGAR). MGAR not only focuses on the semantic alignment of overall descriptions but also delves into the semantic elements in language, particularly nouns and verbs, and aligns them with frame-level and motion-level features of fake news videos for multi-granularity reasoning. Additionally, we design a unified residual-structured multi-granularity language module that employs a context exchange mechanism (e.g., word-level and sentence-level) to adapt to semantic understanding at different granularity. Extensive experiments on the FakeVE dataset demonstrate the superiority of MGAR, achieving improvements of +10.1% BLEU-1 and +11.1% ROUGE-L over state-of-the-art baselines, showcasing the potential of AI applications in combating false information.