Classifier ensemble with relevance-based feature subset selection

Abstract

For classifier ensemble systems, diversity is considered as an important factor for good generalization. In this paper, a classifier ensemble algorithm with relevance-based feature subset selection for classification is proposed. Firstly, a combined maximal class relevance and minimal feature relevance criterion is presented to evaluate candidate features, and to search diverse feature subspaces for classifier ensemble. And then bootstrap sampling is implemented on the feature subspaces for diverse training sets. Finally, the classifiers are trained on diverse data subspaces selected by feature subset search method with subsequent bootstrap. The experimental results show that the algorithm achieves high classification accuracies with small feature subspaces, which lead to a compact and effective ensemble system.