Alexandre Heuillet, Ahmad Nasser, Hichem Arioui, Hedi Tabia
{"title":"Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search","authors":"Alexandre Heuillet, Ahmad Nasser, Hichem Arioui, Hedi Tabia","doi":"10.1145/3665138","DOIUrl":null,"url":null,"abstract":"<p>In the past few years, Differentiable Neural Architecture Search (DNAS) rapidly imposed itself as the trending approach to automate the discovery of deep neural network architectures. This rise is mainly due to the popularity of DARTS (Differentiable ARchitecTure Search), one of the first major DNAS methods. In contrast with previous works based on Reinforcement Learning or Evolutionary Algorithms, DNAS is faster by several orders of magnitude and uses fewer computational resources. In this comprehensive survey, we focused specifically on DNAS and reviewed recent approaches in this field. Furthermore, we proposed a novel challenge-based taxonomy to classify DNAS methods. We also discussed the contributions brought to DNAS in the past few years and its impact on the global NAS field. Finally, we concluded by giving some insights into future research directions for the DNAS field.</p>","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"8 1","pages":""},"PeriodicalIF":23.8000,"publicationDate":"2024-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ACM Computing Surveys","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1145/3665138","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, THEORY & METHODS","Score":null,"Total":0}
引用次数: 0
Abstract
In the past few years, Differentiable Neural Architecture Search (DNAS) rapidly imposed itself as the trending approach to automate the discovery of deep neural network architectures. This rise is mainly due to the popularity of DARTS (Differentiable ARchitecTure Search), one of the first major DNAS methods. In contrast with previous works based on Reinforcement Learning or Evolutionary Algorithms, DNAS is faster by several orders of magnitude and uses fewer computational resources. In this comprehensive survey, we focused specifically on DNAS and reviewed recent approaches in this field. Furthermore, we proposed a novel challenge-based taxonomy to classify DNAS methods. We also discussed the contributions brought to DNAS in the past few years and its impact on the global NAS field. Finally, we concluded by giving some insights into future research directions for the DNAS field.
期刊介绍:
ACM Computing Surveys is an academic journal that focuses on publishing surveys and tutorials on various areas of computing research and practice. The journal aims to provide comprehensive and easily understandable articles that guide readers through the literature and help them understand topics outside their specialties. In terms of impact, CSUR has a high reputation with a 2022 Impact Factor of 16.6. It is ranked 3rd out of 111 journals in the field of Computer Science Theory & Methods.
ACM Computing Surveys is indexed and abstracted in various services, including AI2 Semantic Scholar, Baidu, Clarivate/ISI: JCR, CNKI, DeepDyve, DTU, EBSCO: EDS/HOST, and IET Inspec, among others.