Generative neural architecture search

Abstract

Neural architecture search (NAS) is an important approach for automatic neural architecture design and has been applied to many tasks, such as image classification and object detection. However, most of the conventional NAS algorithms mainly focus on reducing the prohibitive computational cost, while choosing commonly used reinforcement learning (RL), evolutionary algorithm (EA) or gradient-based methods as their search strategy. In this paper, we propose a novel search strategy for NAS, called Generative NAS (GNAS). Specifically, we assume that high-performing convolutional neural networks adhere to a latent distribution, and design a generator to learn this distribution for generating neural architectures. Furthermore, in order to update the generator for better learning the latent distribution, we use the policy gradient and the performance of the generated CNNs on the validation datasets as a reward signal. To evaluate GNAS, we have conducted extensive experiments on the CIFAR-10, SVHN, MNIST, Fashion-MNIST and ImageNet datasets. The results demonstrate the effectiveness of GNAS compared to previous NAS strategies.