Pre-Optimization of High Dimensional Extreme Learning Machine with Cooperative Coevolution

Abstract

Extreme Learning Machine (ELM) is a special type of single hidden layer feedforward neural network, which uses pseudo-inverse to compute weights of the output layer, and is often faster than the gradient-based methods. However, due to the random initialization of input weights and biases, the performance of this algorithm is not always stable, and is less effective in large-scale applications. Therefore, in this paper, based on an effective large-scale optimization algorithm, i.e., cooperative coevolutionary particle swarm optimization (CCPSO), an improved hybrid ELM learning algorithm, named CCPSO-ELM, is proposed, where the input weights and the hidden layer biases are optimized using CCPSO. Compared with the traditional ELM algorithm, as well as ELM optimized by traditional PSO, the CCPSO-ELM is more likely to avoid local optima, has smaller optimization errors, and is more robust against noises. The results are verified by experiments on two different types of problems, i.e., large-scale multivariate function approximation and pattern classification.