Multi-View mutual learning network for multimodal fake news detection

Abstract

Multimodal fake news is more deceptive than unimodal content and often has adverse social and economic impacts. However, most existing methods learn modal features from a single perspective, without considering simultaneously learning and sharing knowledge across modalities from different perspectives. Our work presents a novel Multi-View Mutual Learning (MVML) network for multimodal fake news detection, which explores the semantic relationship between words and scenes, as well as words and objects, and investigates the semantic connection that exists between global concepts and local objects from multiple perspectives. We first construct text-scenes and text-objects graphs respectively, and perform intra-graph inference to obtain multi-view features. Then, the multi-view fusion layer based on the cross-view attention mechanism interactively models the cross-view dependencies between the image and text. The extracted features before feeding into the final classifier are further processed via a feed-forward attention module, which adaptively reweights and aggregates the features for redundancy reduction. In addition, to exploit the respective advantages of two multi-view classifiers, we propose a mutual learning mechanism that allows them to perform knowledge distillation and align the learning targets. The proposed MVML is thoroughly assessed on four publicly available benchmark datasets, and the findings demonstrate that it outperforms the existing standard approaches.