Leveraging facial landmarks improves generalization ability for deepfake detection

Abstract

Recently, facial forgery technology has become increasingly sophisticated and published datasets aim to cover a wide range of data variations. Existing deepfake detection models have benefited from the powerful feature embedding of deep networks and carefully designed fine-tuning modules, resulting in an excellent performance on in-dataset evaluations. However, the performance declines in cross-dataset evaluations due to various forgery methods and dataset shifts. In this study, we concentrate on the generalization issue of deepfake detection and find that forgery traces appear to gather around the facial interest points even manipulated by different forgery methods. To facilitate this, we propose a Trail Tracing Network (TTNet) to capture the generalized feature representation, which leverages facial landmarks to eliminate redundant information and expand the forged traces in the feature space. We conduct extensive experiments on the widely employed benchmarks, including FaceForensics++, DFDCp, and Celeb-DF. Experimental results demonstrate the outstanding generalization ability of our method against existing state-of-the-art methods by a large margin. In addition, the proposed method also exhibits excellent performance on the in-dataset evaluation.