CAMFND: Cross-modal adaptive-aware learning for multimodal fake news detection

Recently, there has been a growing focus on the automatic identification of multimodal fake news detection. A fundamental challenge of multimodal fake news detection lies in the inherent semantic ambiguity across different content modalities. Decisions stemming from distinct unimodal sources may exhibit discrepancies, potentially creating inconsistency with the collective insights derived from multimodal data fusion. To address this issue, we propose CAMFND: a cross-modal adaptive-aware learning framework for multi-modal fake news detection, aiming to reduce semantic ambiguities among different modalities. CAMFND consists of (1) a cross-modal alignment module to transform the heterogeneous unimodality features into a shared semantic space, (2) a cross-modal adaptive-interactive module to capture the semantic correlation and consistency, computed by the multi-modal gated fusion unit, (3) a cross-modal adaptive-selective module to decide the semantic meaning or bias, guided by the multi-modal semantic matching score. CAMFND enhances the fake news detection by intelligently and dynamically combining features from uni-modality and identifying correlations across different modalities. It leverages unimodal features in scenarios with low cross-modal ambiguity, while utilizing cross-modal correlations in cases of high cross-modal uncertainty. The experimental results show that CAMFND significantly surpasses prior methodologies and sets new benchmarks on both English Twitter and Chinese Weibo datasets, marking a notable advancement in performance.