Selecting diverse members of neural network ensembles

Abstract

Ensembles of artificial neural networks have been used as classification/regression machines, showing improved generalization capabilities that outperform those of single networks. However, it has been recognized that for aggregation to be effective the individual network must be as accurate and diverse as possible. An important problem is, then, how to choose the aggregate members in order to have an optimal compromise between these two conflicting conditions. We propose here a new method for selecting members of regression/classification ensembles that leads to small aggregates with few but very diverse individual predictors. Using artificial neural networks as individual learners, the algorithm is favorably tested against other methods in the literature, producing a remarkable performance improvement on the standard statistical databases used as benchmarks. In addition, and as a concrete application, we study the sunspot time series and predict the remaining of the current cycle 23 of solar activity.