Semantic space aligned multimodal fake news detection

Abstract

The spread of fake news misleads the uninformed public into believing incorrect information, which has a negative impact on society. With the majority of current social media content being a combination of images and text, there has been an increasing interest in multimodal fake news detection methods. However, existing multimodal fake news detection methods often overlook the complementary role of consistency at different levels. Moreover, they cannot ensure isotropic calculation space when measuring consistency. This paper proposes SSA-MFND, a multimodal fake news detection model based on semantic space alignment. To fully integrate the information from each modality, the model compares the semantic consistency between the description layer and the entity layer, projecting the semantic representation into isotropic space before the semantic consistency calculation. Additionally, our model leverages semantic consistency comparisons at both the description and entity levels, addressing a critical gap in current multimodal methods that typically consider only one level of consistency. Experimental results show that the model achieves an improved accuracy of 2% compared to the state-of-the-art on the microblog dataset. Incorporating semantic hierarchical information and projecting the semantic space before computation greatly enhances the accuracy of fake news detection. The SSA-MFND model can be effectively applied to social media platforms, aiding in the automated detection of fake information and reducing the spread of misleading content. This application not only helps improve the quality of content on the platform but also protects users from the influence of false information.