A novel sparse ensemble pruning algorithm using a new diversity measure

Abstract

Extreme learning machine is state of art supervised machine learning technique for classification and regression. A single ELM classifier can however generate faulty or skewed results due to random initialization of weights between input and hidden layer. To overcome this instability problem ensemble methods can be employed. Ensemble methods may have problem of redundancy i.e. ensemble may contain several redundant classifiers which can be weak or highly correlated classifiers. Ensemble pruning can be used to remove these redundant classifiers. The pruned ensemble should not only be accurate but diverse as well in order to correctly classify boundary instances. This work proposes an ensemble pruning algorithm which tries to establish a tradeoff between accuracy and diversity. The paper also proposes a metric which scores classifiers based on their diversity and contribution towards the ensemble. The results show that the pruned ensemble performs equally well or in some cases even better as compared to the unpruned set in terms of accuracy and diversity. The results of the experiments show that the proposed algorithm performs better than VELM. The proposed algorithm reduces the ensemble size to less than 60 % of the original ensemble size (original ensemble size is set to 50).