A neural network based on back-propagation and cooperative co-evolution

Deep neural networks (DNNs) have a powerful feature extraction capability, which allows them to be employed in various fields. However, as the number of layers and neurons in the network increases, the search space for parameter learning becomes complex. Currently, the most commonly used parameter training method is backpropagation (BP) based on gradient descent, but this method is sensitive to the initialization of the parameters and tends to get stuck in local optima in a complex search space. Therefore, a new training method for DNNs has been proposed that combines cooperative co-evolution (CC) with BP-based gradient descent, called BPCC. In the BPCC method, BP performs multiple training periods intermittently, and the CC algorithm is executed when the difference between the current loss function value and the previous loss function value is less than a given threshold (called a condition met). We found that the algorithm easily enters into CC iterations, which reduces the computational effectiveness of the algorithm. A tolerance parameter is designed to curb this phenomenon, and the CC is executed when the cumulative number of times the condition is met reaches the given value of the tolerance parameter, and the improved gray wolf optimizer (GWO) algorithm is used as the solver for the CC. In addition, in the CC iteration stage, the Chebyshev chaotic map series based on the current optimal point is used to initialize the population of GWO to ensure the diversity of the initial population. Experimental comparisons are made with modern network training methods in 7 network models, and the experimental results show that the improved algorithm in this study is competitive.