Feature selection using different evaluate strategy and random forests

Abstract

Aiming at the dimensional disaster and over-fitting problems in data analysis, this paper proposes a feature selection method using hybrid integration of difference models and random forests (Integrate-RF), firstly, Integrate-RF use CART, CHAID, SVM, BN, NN, K-Means, Kohonen to evaluate the importance of features, and then, for the above seven sorts, Integrate-RF use the arithmetic average method to calculate the importance of the features; secondly, Integrate-RF select the most important features from the remaining features into features subset, and use random forest classification to get the corresponding out-of-bag(OOB) data classification error rate; finally, the optimal features subset can be selected based on the OOB data classification error rate. Experiments show that feature selection methods proposed in this paper effectively reduces the data dimension, selects features better and more adaptable.