Contrastive Domain Adaptation with Test-Time Training for Out-of-Context News Detection

Abstract

Out-of-context news is a common type of misinformation on online media platforms. This involves posting a caption, alongside a mismatched news image. Reflecting its importance, researchers have developed models to detect such misinformation. However, a common limitation of these models is that they only consider the scenario where pre-labelled data is available for each news topic or agency, failing to address the out-of-context news detection on unverified news of other topics or agencies. In this work, we therefore focus on domain adaptive out-of-context news detection. We regard news topic or news agency as the domain. In order to effectively adapt the detection model to unlabelled news topics or agencies, we propose Contrastive Domain Adaptation with Test-Time Training (ConDA-TTT). It first applies contrastive learning to learn a more separable representation space for news inputs, and then uses maximum mean discrepancy (MMD) to remove the domain-specific features so as to keep the domain-invariant features. During test time, it uses the trained model to predict pseudo labels for the target domain test data, and selects those with higher confidence scores to train the classifier of the model, in order to further adapt the model to the target domain data distribution. This approach adapts the model at both training and test phase, making the domain adaptation more robust to distribution shifts. Experimental results demonstrate that our approach outperforms state-of-the-art baselines in all the domain adaptation settings on two benchmark datasets, by as much as 2.6% in F1 and 2.4% in accuracy.