利用领域可靠的表征学习实现通用人脸伪造检测

IF 2.9 3区 工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC
Caiyu Li, Yan Wo
{"title":"利用领域可靠的表征学习实现通用人脸伪造检测","authors":"Caiyu Li,&nbsp;Yan Wo","doi":"10.1016/j.dsp.2024.104792","DOIUrl":null,"url":null,"abstract":"<div><div>Face forgery detection is crucial for the security of digital identities. However, existing methods often struggle to generalize effectively to unseen domains due to the domain shift between training and testing data. We propose a Domain-robust Representation Learning (DRRL) method for generalized face forgery detection. Specifically, we observe that domain shifts in face forgery detection tasks are often caused by forgery differences and content differences between domain data, while the limitations of training data lead the model to overfit to these feature expressions in the seen domain. Therefore, DRRL enhances the model's generalization to unseen domains by first adding representative data representations to mitigate overfitting to seen data and then removing the features of expressed domain information to learn a robust, discriminative representation of domain variation. Data augmentation is achieved by stylizing sample representations and exploring representative new styles to generate rich data variants, with the Content-style Augmentation (CSA) module and Forgery-style Augmentation (FSA) module implemented for content and forgery expression, respectively. Based on this, the Content Decorrelation (CTD) module and Sensitive Channels Drop (SCD) module are used to remove content features irrelevant to forgery and domain-sensitive forgery features, encouraging the model to focus on clean and robust forgery features, thereby achieving the goal of learning domain-robust representations. Extensive experiments on five large-scale datasets demonstrate that our method exhibits advanced and stable generalization performance in practical scenarios.</div></div>","PeriodicalId":51011,"journal":{"name":"Digital Signal Processing","volume":"156 ","pages":"Article 104792"},"PeriodicalIF":2.9000,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Towards generalized face forgery detection with domain-robust representation learning\",\"authors\":\"Caiyu Li,&nbsp;Yan Wo\",\"doi\":\"10.1016/j.dsp.2024.104792\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Face forgery detection is crucial for the security of digital identities. However, existing methods often struggle to generalize effectively to unseen domains due to the domain shift between training and testing data. We propose a Domain-robust Representation Learning (DRRL) method for generalized face forgery detection. Specifically, we observe that domain shifts in face forgery detection tasks are often caused by forgery differences and content differences between domain data, while the limitations of training data lead the model to overfit to these feature expressions in the seen domain. Therefore, DRRL enhances the model's generalization to unseen domains by first adding representative data representations to mitigate overfitting to seen data and then removing the features of expressed domain information to learn a robust, discriminative representation of domain variation. Data augmentation is achieved by stylizing sample representations and exploring representative new styles to generate rich data variants, with the Content-style Augmentation (CSA) module and Forgery-style Augmentation (FSA) module implemented for content and forgery expression, respectively. Based on this, the Content Decorrelation (CTD) module and Sensitive Channels Drop (SCD) module are used to remove content features irrelevant to forgery and domain-sensitive forgery features, encouraging the model to focus on clean and robust forgery features, thereby achieving the goal of learning domain-robust representations. Extensive experiments on five large-scale datasets demonstrate that our method exhibits advanced and stable generalization performance in practical scenarios.</div></div>\",\"PeriodicalId\":51011,\"journal\":{\"name\":\"Digital Signal Processing\",\"volume\":\"156 \",\"pages\":\"Article 104792\"},\"PeriodicalIF\":2.9000,\"publicationDate\":\"2024-09-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Digital Signal Processing\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1051200424004172\",\"RegionNum\":3,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Digital Signal Processing","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1051200424004172","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

摘要

人脸伪造检测对数字身份安全至关重要。然而,由于训练数据和测试数据之间的领域转移,现有方法往往难以有效地推广到未知领域。我们提出了一种用于通用人脸伪造检测的领域稳健表征学习(DRRL)方法。具体来说,我们观察到人脸伪造检测任务中的域转移通常是由域数据之间的伪造差异和内容差异引起的,而训练数据的局限性导致模型在可见域中过度拟合这些特征表达。因此,DRRL 首先通过添加有代表性的数据表示来减轻对所见数据的过度拟合,然后移除所表达的领域信息特征,从而学习领域变化的稳健性和鉴别性表征,从而增强模型对未见领域的泛化能力。数据增强是通过将样本表示风格化和探索有代表性的新风格来生成丰富的数据变体来实现的,内容风格增强(CSA)模块和伪造风格增强(FSA)模块分别用于内容和伪造表达。在此基础上,利用内容去相关性(Content Decorrelation,CTD)模块和敏感通道去除(Sensitive Channels Drop,SCD)模块去除与伪造无关的内容特征和对领域敏感的伪造特征,促使模型专注于干净、稳健的伪造特征,从而实现学习领域稳健表征的目标。在五个大规模数据集上进行的广泛实验证明,我们的方法在实际应用场景中表现出先进而稳定的泛化性能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Towards generalized face forgery detection with domain-robust representation learning
Face forgery detection is crucial for the security of digital identities. However, existing methods often struggle to generalize effectively to unseen domains due to the domain shift between training and testing data. We propose a Domain-robust Representation Learning (DRRL) method for generalized face forgery detection. Specifically, we observe that domain shifts in face forgery detection tasks are often caused by forgery differences and content differences between domain data, while the limitations of training data lead the model to overfit to these feature expressions in the seen domain. Therefore, DRRL enhances the model's generalization to unseen domains by first adding representative data representations to mitigate overfitting to seen data and then removing the features of expressed domain information to learn a robust, discriminative representation of domain variation. Data augmentation is achieved by stylizing sample representations and exploring representative new styles to generate rich data variants, with the Content-style Augmentation (CSA) module and Forgery-style Augmentation (FSA) module implemented for content and forgery expression, respectively. Based on this, the Content Decorrelation (CTD) module and Sensitive Channels Drop (SCD) module are used to remove content features irrelevant to forgery and domain-sensitive forgery features, encouraging the model to focus on clean and robust forgery features, thereby achieving the goal of learning domain-robust representations. Extensive experiments on five large-scale datasets demonstrate that our method exhibits advanced and stable generalization performance in practical scenarios.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Digital Signal Processing
Digital Signal Processing 工程技术-工程:电子与电气
CiteScore
5.30
自引率
17.20%
发文量
435
审稿时长
66 days
期刊介绍: Digital Signal Processing: A Review Journal is one of the oldest and most established journals in the field of signal processing yet it aims to be the most innovative. The Journal invites top quality research articles at the frontiers of research in all aspects of signal processing. Our objective is to provide a platform for the publication of ground-breaking research in signal processing with both academic and industrial appeal. The journal has a special emphasis on statistical signal processing methodology such as Bayesian signal processing, and encourages articles on emerging applications of signal processing such as: • big data• machine learning• internet of things• information security• systems biology and computational biology,• financial time series analysis,• autonomous vehicles,• quantum computing,• neuromorphic engineering,• human-computer interaction and intelligent user interfaces,• environmental signal processing,• geophysical signal processing including seismic signal processing,• chemioinformatics and bioinformatics,• audio, visual and performance arts,• disaster management and prevention,• renewable energy,
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信