{"title":"Local artifacts amplification for deepfakes augmentation","authors":"","doi":"10.1016/j.neunet.2024.106692","DOIUrl":null,"url":null,"abstract":"<div><p>With the rapid and continuous development of AIGC, It is becoming increasingly difficult to distinguish between real and forged facial images, which calls for efficient forgery detection systems. Although many detection methods have noticed the importance of local artifacts, there has been a lack of in-depth discussion regarding the selection of locations and their effective utilization. Besides, the traditional image augmentation methods that are widely used have limited improvements for forgery detection tasks and require more specialized augmentation methods specifically designed for forgery detection tasks. In this paper, this study proposes Local Artifacts Amplification for Deepfakes Augmentation, which amplifies the local artifacts on the forged faces. Furthermore, this study incorporates prior knowledge about similar facial features into the model. This means that within the facial regions defined in this work, forged features exhibit similar patterns. By aggregating the results from all facial regions, the study can enhance the overall performance of the model. The evaluation experiments conducted in this research, achieving an AUC of 93.40% and an Acc of 87.03% in the challenging WildDeepfake dataset, demonstrate a promising improvement in accuracy compared to traditional image augmentation methods and achieve superior performance on intra-dataset evaluation. The cross-dataset evaluation also showed that the method presented in this study has strong generalization abilities.</p></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0000,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neural Networks","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0893608024006166","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
With the rapid and continuous development of AIGC, It is becoming increasingly difficult to distinguish between real and forged facial images, which calls for efficient forgery detection systems. Although many detection methods have noticed the importance of local artifacts, there has been a lack of in-depth discussion regarding the selection of locations and their effective utilization. Besides, the traditional image augmentation methods that are widely used have limited improvements for forgery detection tasks and require more specialized augmentation methods specifically designed for forgery detection tasks. In this paper, this study proposes Local Artifacts Amplification for Deepfakes Augmentation, which amplifies the local artifacts on the forged faces. Furthermore, this study incorporates prior knowledge about similar facial features into the model. This means that within the facial regions defined in this work, forged features exhibit similar patterns. By aggregating the results from all facial regions, the study can enhance the overall performance of the model. The evaluation experiments conducted in this research, achieving an AUC of 93.40% and an Acc of 87.03% in the challenging WildDeepfake dataset, demonstrate a promising improvement in accuracy compared to traditional image augmentation methods and achieve superior performance on intra-dataset evaluation. The cross-dataset evaluation also showed that the method presented in this study has strong generalization abilities.
期刊介绍:
Neural Networks is a platform that aims to foster an international community of scholars and practitioners interested in neural networks, deep learning, and other approaches to artificial intelligence and machine learning. Our journal invites submissions covering various aspects of neural networks research, from computational neuroscience and cognitive modeling to mathematical analyses and engineering applications. By providing a forum for interdisciplinary discussions between biology and technology, we aim to encourage the development of biologically-inspired artificial intelligence.