Face manipulated deepfake generation and recognition approaches: a survey

IF 2.4 Q2 MULTIDISCIPLINARY SCIENCES
Mansi Rehaan, Nirmal Kaur, Staffy Kingra
{"title":"Face manipulated deepfake generation and recognition approaches: a survey","authors":"Mansi Rehaan, Nirmal Kaur, Staffy Kingra","doi":"10.1080/23080477.2023.2268380","DOIUrl":null,"url":null,"abstract":"ABSTRACTWith the progression of deep-learning techniques, digital media recording and synthesis media generation have become exceptionally easy. Due to open access of user-friendly deepfaking applications generated using Artificial Intelligence methods especially Generative Adversarial Networks (GANs) and Variational Auto-encoders (VAEs), synthesis of recorded media has been made more effortless. Digital media synthesized using such applications is termed as deepfake. Generation of realistic fake audio/video content poses critical threats to an individual and society at large. To curb the threat of deepfaking, numerous deepfake detection algorithms have been proposed. This paper presents a survey on state-of-the-art deepfake generation techniques, categorized into face swap, attribute manipulation, and lip-sync manipulation. An analysis of some recently developed techniques that can generate images from text is also presented in the proposed paper. In addition, state-of-art image and video datasets released in the domain of deepfake detection are also discussed. The analysis provided in this survey reveals that every new deepfake generation technique calls for the development of novel deepfake detection techniques. Among deepfake detection techniques proposed so far, MobileNet-CNN model, multi-scale temporal CNN model, and GAN-based technique outperform good for face swap, lip-syncing, and attribute manipulation detection with an accuracy of 99.28%, 97.1% ,and 100%, respectively. This survey would help researchers to understand the literature on deepfake generation and detection which is required for future development in this field.KEYWORDS: Face swapattribute manipulationlip-syncingdeepfake Datasets Disclosure statementNo potential conflict of interest was reported by the author(s).","PeriodicalId":53436,"journal":{"name":"Smart Science","volume":null,"pages":null},"PeriodicalIF":2.4000,"publicationDate":"2023-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Smart Science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/23080477.2023.2268380","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MULTIDISCIPLINARY SCIENCES","Score":null,"Total":0}
引用次数: 0

Abstract

ABSTRACTWith the progression of deep-learning techniques, digital media recording and synthesis media generation have become exceptionally easy. Due to open access of user-friendly deepfaking applications generated using Artificial Intelligence methods especially Generative Adversarial Networks (GANs) and Variational Auto-encoders (VAEs), synthesis of recorded media has been made more effortless. Digital media synthesized using such applications is termed as deepfake. Generation of realistic fake audio/video content poses critical threats to an individual and society at large. To curb the threat of deepfaking, numerous deepfake detection algorithms have been proposed. This paper presents a survey on state-of-the-art deepfake generation techniques, categorized into face swap, attribute manipulation, and lip-sync manipulation. An analysis of some recently developed techniques that can generate images from text is also presented in the proposed paper. In addition, state-of-art image and video datasets released in the domain of deepfake detection are also discussed. The analysis provided in this survey reveals that every new deepfake generation technique calls for the development of novel deepfake detection techniques. Among deepfake detection techniques proposed so far, MobileNet-CNN model, multi-scale temporal CNN model, and GAN-based technique outperform good for face swap, lip-syncing, and attribute manipulation detection with an accuracy of 99.28%, 97.1% ,and 100%, respectively. This survey would help researchers to understand the literature on deepfake generation and detection which is required for future development in this field.KEYWORDS: Face swapattribute manipulationlip-syncingdeepfake Datasets Disclosure statementNo potential conflict of interest was reported by the author(s).
人脸操纵深度生成和识别方法:一项调查
摘要随着深度学习技术的发展,数字媒体记录和合成媒体生成变得异常容易。由于使用人工智能方法(特别是生成对抗网络(gan)和变分自编码器(VAEs))生成的用户友好的深度应用程序的开放访问,录制媒体的合成变得更加轻松。使用这些应用程序合成的数字媒体被称为深度造假。产生逼真的假音频/视频内容对个人和整个社会构成严重威胁。为了遏制深度伪造的威胁,已经提出了许多深度伪造检测算法。本文介绍了最先进的深度假生成技术,分为人脸交换、属性操作和口型操作。本文还分析了最近开发的一些从文本生成图像的技术。此外,还讨论了深度伪造检测领域最新发布的图像和视频数据集。本调查提供的分析表明,每一种新的深度假信号生成技术都需要新的深度假信号检测技术的发展。在目前提出的深度伪造检测技术中,MobileNet-CNN模型、多尺度时态CNN模型和基于gan的技术在人脸交换、假声和属性操纵检测方面表现较好,准确率分别为99.28%、97.1%和100%。这一调查将有助于研究人员了解关于深度假信号生成和检测的文献,这是该领域未来发展所需要的。关键词:人脸交换、属性操纵、假唱、deepfake数据集披露声明作者未报告潜在利益冲突。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Smart Science
Smart Science Engineering-Engineering (all)
CiteScore
4.70
自引率
4.30%
发文量
21
期刊介绍: Smart Science (ISSN 2308-0477) is an international, peer-reviewed journal that publishes significant original scientific researches, and reviews and analyses of current research and science policy. We welcome submissions of high quality papers from all fields of science and from any source. Articles of an interdisciplinary nature are particularly welcomed. Smart Science aims to be among the top multidisciplinary journals covering a broad spectrum of smart topics in the fields of materials science, chemistry, physics, engineering, medicine, and biology. Smart Science is currently focusing on the topics of Smart Manufacturing (CPS, IoT and AI) for Industry 4.0, Smart Energy and Smart Chemistry and Materials. Other specific research areas covered by the journal include, but are not limited to: 1. Smart Science in the Future 2. Smart Manufacturing: -Cyber-Physical System (CPS) -Internet of Things (IoT) and Internet of Brain (IoB) -Artificial Intelligence -Smart Computing -Smart Design/Machine -Smart Sensing -Smart Information and Networks 3. Smart Energy and Thermal/Fluidic Science 4. Smart Chemistry and Materials
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信