Stochastic Induction of Decision Trees with Application to Learning Haar Trees

Abstract

Decision trees are a convenient and established approach for any supervised learning task. Decision trees are trained by greedily splitting a leaf nodes, into two leaf nodes until a specific stopping criterion is reached. Splitting a node consists of finding the best feature and threshold that minimizes a criterion. The criterion minimization problem is solved through a costly exhaustive search algorithm. This paper proposes a novel stochastic approach for criterion minimization. The algorithm is compared with several other related state-of-the-art decision tree learning methods, including the baseline non-stochastic approach. We apply the proposed algorithm to learn a Haar tree over MNIST dataset that consists of over 200, 000 features and 60, 000 samples. The result is comparable to the performance of oblique trees while providing a significant speed-up in both inference and training times.