Identifying Rhythmic Patterns for Face Forgery Detection and Categorization

With the emergence of GAN, face forgery technologies have been heavily abused. Achieving accurate face forgery detection is imminent. Inspired by remote photoplethysmography (rPPG) that PPG signal corresponds to the periodic change of skin color caused by heartbeat in face videos, we observe that despite the inevitable loss of PPG signal during the forgery process, there is still a mixture of PPG signals in the forgery video with a unique rhythmic pattern depending on its generation method. Motivated by this key observation, we propose a two-stage network for face forgery detection and categorization consisting of: 1) a Spatial-Temporal Filter Module (STFM) for PPG signals filtering, and 2) an Adjacency Interaction Module (AIM) for constraint and interaction of PPG signals. Moreover, with insight into the generation of forgery methods, we further propose Spatial-Temporal Mixup (ST-Mixup) to boost the performance of the network. Overall, extensive experiments have proved the superiority of our method.