Jicheng Li , Yongjian Hu , Beibei Liu , Huimin She , Chang-Tsun Li
{"title":"Deepfake detection with domain generalization and mask-guided supervision","authors":"Jicheng Li , Yongjian Hu , Beibei Liu , Huimin She , Chang-Tsun Li","doi":"10.1016/j.patcog.2025.111622","DOIUrl":null,"url":null,"abstract":"<div><div>Most existing deepfake (video face forgery) detectors work well in intra-dataset testing, but their performance degrades severely in cross-dataset testing. Cross-dataset generalization remains a major challenge. Since domain generalization (DG) aims to learn domain-invariant features while suppressing domain specific features, we propose a DG framework for improving face forgery detection in this study. Our detector consists of two modules. The first module learns both spatial and spectral features from frame images. The second one learns high-level feature patterns from the outputs of the first module, and constructs the classification features with the help of face mask-guided supervision. The classification result is fine-tuned by a confidence-based correction mechanism. The DG framework is realized through a bi-level optimization process. Extensive experiments demonstrate that our detector works effectively in both intra- and cross-dataset testing. Compared with 8 typical methods, it has the best overall performance and the highest robustness against common perturbations.</div></div>","PeriodicalId":49713,"journal":{"name":"Pattern Recognition","volume":"165 ","pages":"Article 111622"},"PeriodicalIF":7.5000,"publicationDate":"2025-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Pattern Recognition","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0031320325002821","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Most existing deepfake (video face forgery) detectors work well in intra-dataset testing, but their performance degrades severely in cross-dataset testing. Cross-dataset generalization remains a major challenge. Since domain generalization (DG) aims to learn domain-invariant features while suppressing domain specific features, we propose a DG framework for improving face forgery detection in this study. Our detector consists of two modules. The first module learns both spatial and spectral features from frame images. The second one learns high-level feature patterns from the outputs of the first module, and constructs the classification features with the help of face mask-guided supervision. The classification result is fine-tuned by a confidence-based correction mechanism. The DG framework is realized through a bi-level optimization process. Extensive experiments demonstrate that our detector works effectively in both intra- and cross-dataset testing. Compared with 8 typical methods, it has the best overall performance and the highest robustness against common perturbations.
期刊介绍:
The field of Pattern Recognition is both mature and rapidly evolving, playing a crucial role in various related fields such as computer vision, image processing, text analysis, and neural networks. It closely intersects with machine learning and is being applied in emerging areas like biometrics, bioinformatics, multimedia data analysis, and data science. The journal Pattern Recognition, established half a century ago during the early days of computer science, has since grown significantly in scope and influence.