Determining the Neural Network Topology: A Review

Abstract

One of the challenges in the successful implementation of deep neural network (DNN) is setting the value for various hyper-parameters, one of which is the network topology, which is closely related to the number of hidden layers and the number of hidden neurons. Determining the number of hidden layers and the number of neurons is very important and has a large influence on DNN performance. Determining these two numbers manually (usually through trial and error methods) to find fairly optimal arrangement is a time-consuming process, while the automatic approach is divided into two, they are a model-based approach and a non-model based approach. The non-model-based approach, for example, is grid search and random search, whereas model-based approaches, for example, are using particle swarm optimization (PSO) algorithms. In some researches, how to determine the number of hidden layers or number of neurons, often the guidelines are unclear, even the roles and functions of both are explained minimally. Although it is still a difficult area of research, research to determine the number of hidden layers and the number of neurons must continue to be carried out, because these two numbers will greatly determine the performance of DNN.