Ensemble of Extreme Learning Machines for Regression

Abstract

Regression, as a particular task of machine learning, performs a vital part in data-driven modeling, by finding the connections between the system state variables without any explicit knowledge about the system, using a collection of input-output data. To enhance the prediction performance and maximize the training speed, we propose a fully learnable ensemble of Extreme Learning Machines (ELMs) for regression. The developed approach learns the combination of different individual models, using the ELM algorithm, which is applied to minimize both the prediction error and the norm of the network parameters, which leads to higher generalization performance under Bartlett’s theory. Moreover, the average based ELM ensemble may be viewed as a particular case of our model. Extensive experiments on many standard regression benchmark datasets have been carried out, and comparison with different models has been performed. The experimental findings confirm that the proposed ensemble can reach competitive results in term of the generalization performance, and the training speed. Furthermore, the influence of different hyperparameters on the performance, in term of the prediction error and the training speed, of the developed model has been investigated to provide a meaningful guideline to practical applications.