A Novel Bayesian Approach for Construction of Random Forest

Abstract

Decision tree is one of the commonly used machine learning algorithm. Random Forest (RF) is an ensemble of such decision trees. The construction of optimal Decision Tree and hence Random Forest is NP Hard when data is large. The Bayesian statistics have been used in the past for various machine learning and pattern recognition problems. The Bayesian statistics provide a tool to construct Random Forest when no prior information for data is available. Here a forest is generated based on Bayesian statistics where numerous trees are sampled given the prior distribution without the use of training data, and after that weighted ensemble is performed. In the past, it has been used for classification problems. In this paper, we are proposing construction of RF under Bayesian framework using Tree Strength concept. Also, we extend our proposal to regression problems. The proposal is evaluated on UCI data sets for both classification and regression task and found satisfactory results.