Centroid-based Contrastive Consistency Learning for transferable deepfake detection

Abstract

Previous research efforts in deepfake detection mainly concentrated on identifying and differentiating artifacts discernible to humans. Those methods have left a bias in learned models, as they tend to concentrate on the disparities between forged and natural regions from the perspective of a single sample while overlooking consistency within categories from the perspective of the entire sample set, which remains crucial across various real-world applications. Therefore, inspired by contrastive learning, we tackle the deepfake detection problem by learning the invariant representations of both categories. Our proposed method, termed Centroid-based Contrastive Consistency Learning (C3L) method, integrates constraints on representations at both the data preprocessing and feature extraction stages. Specifically, during data preprocessing, we consider both temporal relationships within videos and the latent relationships within synthesis data. We introduce a novel Positive Enhancement Module (PEM) designed to characterize natural and forged samples in a facial semantically irrelevant way, thereby guiding a task-oriented positive pair contrasting strategy. In addition, at the feature extraction stage, we introduce the Margin Feature Simulation Module (MFSM), which leverages the centroid of the natural category to simulate marginal features for both categories. Subsequently, we employ the Supervised Contrastive Margin Loss (SCML), utilizing simulated features to emphasize differences at decision boundaries and optimize the learning process. The effectiveness and robustness of the proposed method have been demonstrated through extensive experiments.