Efficient configuration of high-dimensional hyperparameters in deep convolutional neural networks for classification assisted by surrogate models

Abstract

The rationality of hyperparameter configuration in deep convolutional neural networks for classification directly determines its performance. It is challenging to reduce high computational costs effectively and guarantee performance in the configuration of high-dimensional hyperparameters in deep convolutional neural networks for classification. This paper proposes an efficient configuration method that concerns high-dimensional hyperparameters in deep convolutional neural networks for classification assisted by surrogate models. By designing a progressive accumulation dropout neural network surrogate model (PA-Dropout), the contribution of hyperparameters configurations to multi-performance objectives is dynamically measured and then the contribution is iteratively screened. As a result, the fitting efficiency of the PA-Dropout to the relationship between high-dimensional hyperparametric configurations and the multi-objective performance in deep convolutional neural networks for classification with scarce data is improved. A dual-drive interactive dynamic model management strategy (DDIDMMS) is designed, considering the comprehensive evaluation and adaptive weighting calculation of convergence diversity of high-dimensional hyperparametric configuration individuals. Reliable candidate solutions are provided for real evaluation, thereby improving the update efficiency of PA-Dropout. Finally, an efficient configuration of high-dimensional hyperparameters in deep convolutional neural networks for classification is realized. By using DTLZ and WFG benchmark problems with up to 100 decision variables and 20 targets, as well as practical classification tasks, the superiority and generalization of this method are verified when solving the expensive multi-objective optimization problem of CNN high-dimensional hyperparameter configuration for classification tasks.