Online incremental random forests

Abstract

In this paper, we propose online method for generating relevant feature incrementally to be learned simultaneously with random forests algorithm. The algorithm iteratively estimates the importance of variables and selects them accordingly based on correlation ranking. We test our method by sequential forward/backward selection approach. Empirical comparisons with 3 other state-of-the-art batch mode features selection approaches (Gini index, ReliefF, Gain ratio) are very encouraging. Using 12 UCI datasets we demonstrate experimentally that that our online methods prediction performs comparably to other batch learning counterpart algorithms.