Work-in-Progress: SuperNAS: Fast Multi-Objective SuperNet Architecture Search for Semantic Segmentation

Abstract

We present SuperNAS, a fast multi-objective neural architecture search framework for semantic segmentation. SuperNAS subsamples the structure and pre-trained parameters of DeepLabV3+, without fine-tuning, dramatically reducing training time during search. To further reduce candidate evaluation time, we use a subset of the validation dataset during search. Only the final, Pareto-dominant, candidates are ultimately fine-tuned using the complete training set. We evaluate SuperNAS by searching for models that effectively trade accuracy and computational cost on the PASCAL VOC 2012 dataset. SuperNAS finds competitive designs quickly, e.g., taking just 0.5 GPU days to discover a DeepLabV3+ variant that reduces FLOPs and parameters by 10% and 20% respectively, for less than 3% increased error.