ID3: Identity-Driven Learning Based on 3D Reconstruction and Frame-Level Residual Enhancement for Deepfakes Detection

Abstract

With the rapid advancement of face manipulation technology, various forged videos of celebrities and politicians have appeared and cause pernicious social impact. In this light, forge video detection becomes a research hot spot recently. Most previous detection approaches focus mainly on forgery artifacts caused by the specific generation defects without considering the individual identity information, so that the detection accuracy is not satisfactory. For instance, for a forgery video of a certain celebrity, everyone knows who she/he is, while this important identity clue is not utilized in current detection methods. To address this problem, a novel perspective of face forgery detection via identity-driven learning, named Identity-Driven Deepfakes Detection (ID³), is proposed. By the proposed method, the similarity between suspect inputs and the inherent properties (e.g., geometry and appearance) of the same identity is considered and explored. Specifically, by 3D reconstruction, the physical differences between the forged and real videos are captured in the learning process. In addition, with frame level residual enhancement, the detection accuracy can be further improved. The validity of the proposed method is experimentally verified on several benchmark datasets, and our detection performance is better than some state-of-the-art works.