Boosting Deepfake Detection Generalizability via Expansive Learning and Confidence Judgement

Abstract

As deepfake technology poses severe threats to information security, significant efforts have been devoted to deepfake detection. To enable model generalization for detecting new types of deepfakes, it is required that the existing models should learn knowledge about new types of deepfakes without losing prior knowledge, a challenge known as catastrophic forgetting (CF). Existing methods mainly utilize domain adaptation to learn about the new deepfakes for addressing this issue. However, these methods are constrained to utilizing a small portion of data samples from the new deepfakes, and they suffer from CF when the size of the data samples used for domain adaptation increases. This resulted in poor average performance in source and target domains. In this paper, we introduce a novel approach to boost the generalizability of deepfake detection. Our approach follows a two-stage training process: training in the source domain (prior deepfakes that have been used for training) and domain adaptation to the target domain (new types of deepfakes). In the first stage, we employ expansive learning to train our expanded model from a well-trained teacher model. In the second stage, we transfer the expanded model to the target domain while removing assistant components. For model architecture, we propose the frequency extraction module to extract frequency features as complementary to spatial features and introduce spatial-frequency contrastive loss to enhance feature learning ability. Moreover, we develop a confidence judgement module to eliminate conflicts between new and prior knowledge. Experimental results demonstrate that our method can achieve better average accuracy in source and target domains even when using large-scale data samples of the target domain, and it exhibits superior generalizability compared to state-of-the-art methods.