Online random forests based on CorrFS and CorrBE

Abstract

This paper aims to contribute to the merits of online ensemble learning for classification problems. To this end we induce random forests algorithm into online mode and estimate the importance of variables incrementally based on correlation ranking (CR). We test our method by an ldquoincremental hill climbingrdquo algorithm in which features are greedily added in a ldquoforwardrdquo step (FS), and removed in a ldquobackwardrdquo step (BE). We resort to an implementation that combine CR with FS and BE. We call this implementation CorrFS and CorrBE respectively. Evaluation based on public UCI databases demonstrates that our method can achieve comparable performance to classifiers constructed from batch training. In addition, the framework allows a fair comparison among other batch mode feature selection approaches such as Gini index, ReliefF and gain ratio.