Optimization of Classification Rules and Voting Strategies for Random Forest

Abstract

As an efficient learning method, random forest is widely used in data mining, machine learning, artificial intelligence and other fields. It has excellent capabilities in specific practice. However, the decision tree model used in the classification process for random forest traverses all attribute values to find the split points, which leads to over-fitting and reduction of algorithm efficiency. In addition, the meta-base models of random forests vote with the same weight, which may result in decreasing algorithm accuracy. In this paper we accomplish the following two optimization tasks. Firstly, the continuous attributes are discretized based on the boundary theorem of Fayyad and Irani. Secondly, Gaussian mixture model is used to adjust the weight of the meta-base models in optimized random forest according to the similarity between the subsets and the training sets. Finally, the optimized algorithm is applied to the student information data set and the terrain types data set. The experiment results show that the optimized algorithm can effectively improve the classification efficiency and prediction accuracy.