Toward Automated Deep Learning: Advances and Challenges in Neural Architecture Search

Abstract

The remarkable success of deep learning across computer vision, natural language processing, and medical diagnosis has largely depended on manually designed neural architectures—a labor-intensive process lacking transferability. This has motivated the development of automated neural architecture search (NAS). This review systematically examines NAS through four methodological pillars: search space design, architecture optimization, hyperparameter optimization, and performance evaluation. For each pillar, we analyze representative approaches, identify trade-offs (e.g., efficiency vs. reliability), and synthesize key insights, including the dominance of cell-based spaces, the emerging trend of hybridization among optimization strategies, and the synergistic gains of multi-fidelity hyperparameter methods. Beyond these pillars, we discuss quantum NAS as an emerging paradigm and outline seven open challenges, including benchmark expansion, interpretability, human bias, and adversarial robustness. This review provides a comprehensive reference for researchers and practitioners advancing automated deep learning.