Multi-Objective Optimization for Neural Network Structure

Abstract

This study presents a new algorithm for training flexible perceptron multilayer neural networks. This algorithm is based on the multi-objective evolutionary optimization and tries to find the smallest optimal structure simultaneously by reducing the network error. In this method, a compatibility is established between the mean squared error and the vector length of the parameters of the activation functions by using flexible neurons which cause a greater degree of freedom leading to a faster convergence of the neural network. Then, the network structure decreases and the problem of overfitting and local minimum is prevented based on the values of these parameters and the use of the integration method of neurons. Moreover, it increases the power of the generalizability of the neural network. This method was used for classification problems, and the results were compared with AMGA, BCPA, LASSO, and Early Stopping methods. Based on the results, the algorithm proposed in this study usually works better compared to the similar algorithms. In addition, the proposed algorithm is a systematic method for finding the optimal neural network structure.