Exploring coordinated motion patterns of facial landmarks for deepfake video detection

Abstract

Due to the rich geometric and motion information they contain, recent studies indicate that facial landmark clues have significant potential for deepfake video detection. In this paper, we make a key observation that there exist coordinated motions among different facial landmarks for real individuals. While the forgery methods focus more on appearance realism, thus likely to disrupt the underlying coordinated motion patterns. Inspired by this observation, this paper explores how to leverage coordinated motion patterns among facial landmarks to enhance deepfake detection. First, we introduce a coordinated motion landmarks mining strategy (CMLMS), to effectively identify correlated landmarks. Utilizing these correlations, we propose a landmark temporal dynamic relation module (LTDRM), which focuses on the coordinated motion patterns between landmarks while extracting their spatiotemporal features. Specifically, LTDRM constructs an adjacency matrix based on the correlated landmarks and uses graph convolution to selectively aggregate information between correlated landmarks. Additionally, LTDRM is a plug-and-play module and can boost the performance of existing deepfake detection methods with minimal computational overhead. Experimental results validate the effectiveness and generalizability of our method.