{"title":"Semi-supervised accuracy predictor-based multi-objective neural architecture search","authors":"","doi":"10.1016/j.neucom.2024.128472","DOIUrl":null,"url":null,"abstract":"<div><p>The rise of neural architecture search (NAS) demonstrates the deep exploration between the neural network architecture and its performance (e.g., accuracy). Many NAS methods are inefficient because they train all candidates from scratch to obtain their accuracies. Although predictor-based NAS algorithms have been vigorously developed to efficiently and accurately evaluate the performance of candidate architectures, the training of accuracy predictors still require hundreds of architectures with ground truth. To overcome this shortcoming, this paper investigates an evolutionary-based NAS method, which constructs a semi-supervised accuracy predictor to efficiently and accurately evaluate candidate architectures. A one-time extractor and strong regressors are implemented to further enhance the prediction performance of the semi-supervised accuracy predictor. Furthermore, a multi-objective approach is developed to find architectures with high ground truth in a tradeoff between high prediction accuracy and prediction confidence. Experimental results demonstrate the strong competitiveness of the proposed approach on NAS benchmarks. The code is available at <span><span>https://github.com/outofstyle/SAPMNAS</span><svg><path></path></svg></span>.</p></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5000,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neurocomputing","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0925231224012438","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
The rise of neural architecture search (NAS) demonstrates the deep exploration between the neural network architecture and its performance (e.g., accuracy). Many NAS methods are inefficient because they train all candidates from scratch to obtain their accuracies. Although predictor-based NAS algorithms have been vigorously developed to efficiently and accurately evaluate the performance of candidate architectures, the training of accuracy predictors still require hundreds of architectures with ground truth. To overcome this shortcoming, this paper investigates an evolutionary-based NAS method, which constructs a semi-supervised accuracy predictor to efficiently and accurately evaluate candidate architectures. A one-time extractor and strong regressors are implemented to further enhance the prediction performance of the semi-supervised accuracy predictor. Furthermore, a multi-objective approach is developed to find architectures with high ground truth in a tradeoff between high prediction accuracy and prediction confidence. Experimental results demonstrate the strong competitiveness of the proposed approach on NAS benchmarks. The code is available at https://github.com/outofstyle/SAPMNAS.
神经架构搜索(NAS)的兴起证明了神经网络架构与其性能(如准确率)之间的深入探索。许多 NAS 方法效率低下,因为它们要从头开始训练所有候选者以获得准确率。虽然基于预测器的 NAS 算法已被大力开发,以高效、准确地评估候选架构的性能,但准确率预测器的训练仍需要数百个具有地面实况的架构。为了克服这一缺陷,本文研究了一种基于进化的 NAS 方法,该方法构建了一个半监督精度预测器,以高效、准确地评估候选架构。为了进一步提高半监督准确度预测器的预测性能,本文采用了一次性提取器和强回归器。此外,还开发了一种多目标方法,以便在高预测精度和预测置信度之间权衡,找到具有高地面真实度的架构。实验结果表明,所提出的方法在 NAS 基准上具有很强的竞争力。代码见 https://github.com/outofstyle/SAPMNAS。
期刊介绍:
Neurocomputing publishes articles describing recent fundamental contributions in the field of neurocomputing. Neurocomputing theory, practice and applications are the essential topics being covered.