MRAN: Multimodal relationship-aware attention network for fake news detection

Abstract

Existing multimodal fake news detection methods face challenges in jointly capturing the intramodality and cross-modal correlation relationships between image regions and text fragments. Additionally, these methods lack comprehensive hierarchical semantics mining for text. These limitations result in ineffective utilization of multimodal information and impact detection performance. To address these issues, we propose a multimodal relationship-aware attention network (MRAN), which consists of three main steps. First, a multi-level encoding network is employed to extract hierarchical semantic feature representations of text, while the visual feature extractor VGG19 learns image feature representations. Second, the captured text and image representations are input into the relationship-aware attention network, which generates high-order fusion features by calculating the similarity between information segments within modalities and cross-modal similarity. Finally, the fusion features are passed through a fake news detector, which identifies fake news. Experimental results on three benchmark datasets demonstrate the effectiveness of MRAN, highlighting its strong detection performance.