{"title":"Face manipulated deepfake generation and recognition approaches: a survey","authors":"Mansi Rehaan, Nirmal Kaur, Staffy Kingra","doi":"10.1080/23080477.2023.2268380","DOIUrl":null,"url":null,"abstract":"ABSTRACTWith the progression of deep-learning techniques, digital media recording and synthesis media generation have become exceptionally easy. Due to open access of user-friendly deepfaking applications generated using Artificial Intelligence methods especially Generative Adversarial Networks (GANs) and Variational Auto-encoders (VAEs), synthesis of recorded media has been made more effortless. Digital media synthesized using such applications is termed as deepfake. Generation of realistic fake audio/video content poses critical threats to an individual and society at large. To curb the threat of deepfaking, numerous deepfake detection algorithms have been proposed. This paper presents a survey on state-of-the-art deepfake generation techniques, categorized into face swap, attribute manipulation, and lip-sync manipulation. An analysis of some recently developed techniques that can generate images from text is also presented in the proposed paper. In addition, state-of-art image and video datasets released in the domain of deepfake detection are also discussed. The analysis provided in this survey reveals that every new deepfake generation technique calls for the development of novel deepfake detection techniques. Among deepfake detection techniques proposed so far, MobileNet-CNN model, multi-scale temporal CNN model, and GAN-based technique outperform good for face swap, lip-syncing, and attribute manipulation detection with an accuracy of 99.28%, 97.1% ,and 100%, respectively. This survey would help researchers to understand the literature on deepfake generation and detection which is required for future development in this field.KEYWORDS: Face swapattribute manipulationlip-syncingdeepfake Datasets Disclosure statementNo potential conflict of interest was reported by the author(s).","PeriodicalId":53436,"journal":{"name":"Smart Science","volume":null,"pages":null},"PeriodicalIF":2.4000,"publicationDate":"2023-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Smart Science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/23080477.2023.2268380","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MULTIDISCIPLINARY SCIENCES","Score":null,"Total":0}
引用次数: 0
Abstract
ABSTRACTWith the progression of deep-learning techniques, digital media recording and synthesis media generation have become exceptionally easy. Due to open access of user-friendly deepfaking applications generated using Artificial Intelligence methods especially Generative Adversarial Networks (GANs) and Variational Auto-encoders (VAEs), synthesis of recorded media has been made more effortless. Digital media synthesized using such applications is termed as deepfake. Generation of realistic fake audio/video content poses critical threats to an individual and society at large. To curb the threat of deepfaking, numerous deepfake detection algorithms have been proposed. This paper presents a survey on state-of-the-art deepfake generation techniques, categorized into face swap, attribute manipulation, and lip-sync manipulation. An analysis of some recently developed techniques that can generate images from text is also presented in the proposed paper. In addition, state-of-art image and video datasets released in the domain of deepfake detection are also discussed. The analysis provided in this survey reveals that every new deepfake generation technique calls for the development of novel deepfake detection techniques. Among deepfake detection techniques proposed so far, MobileNet-CNN model, multi-scale temporal CNN model, and GAN-based technique outperform good for face swap, lip-syncing, and attribute manipulation detection with an accuracy of 99.28%, 97.1% ,and 100%, respectively. This survey would help researchers to understand the literature on deepfake generation and detection which is required for future development in this field.KEYWORDS: Face swapattribute manipulationlip-syncingdeepfake Datasets Disclosure statementNo potential conflict of interest was reported by the author(s).
期刊介绍:
Smart Science (ISSN 2308-0477) is an international, peer-reviewed journal that publishes significant original scientific researches, and reviews and analyses of current research and science policy. We welcome submissions of high quality papers from all fields of science and from any source. Articles of an interdisciplinary nature are particularly welcomed. Smart Science aims to be among the top multidisciplinary journals covering a broad spectrum of smart topics in the fields of materials science, chemistry, physics, engineering, medicine, and biology. Smart Science is currently focusing on the topics of Smart Manufacturing (CPS, IoT and AI) for Industry 4.0, Smart Energy and Smart Chemistry and Materials. Other specific research areas covered by the journal include, but are not limited to: 1. Smart Science in the Future 2. Smart Manufacturing: -Cyber-Physical System (CPS) -Internet of Things (IoT) and Internet of Brain (IoB) -Artificial Intelligence -Smart Computing -Smart Design/Machine -Smart Sensing -Smart Information and Networks 3. Smart Energy and Thermal/Fluidic Science 4. Smart Chemistry and Materials