A node pruning algorithm for feedforward neural network based on neural complexity

Abstract

In this paper, a hidden node pruning algorithm based on the neural complexity is proposed, the entropy of neural network can be calculated by the standard covariance matrix of the neural network's connection matrix in the training stage, and the neural complexity can be acquired. In ensuring the information processing capacity of neural network is not reduced, select and delete the least important hidden node, and the simpler neural network architecture is achieved. It is not necessary to train the cost function of the neural network to a local minimal, and the pre-processing neural network weights is avoided before neural network architecture adjustment. The simulation results of the non-linear function approximation shows that the performance of the approximation is ensured and at the same time a simple architecture of neural networks can be achieved.