Design and Development of Hybrid Feature Selection Method for Classification

Abstract

To enhance the capability of the learning model in this research paper we have developed a hybrid feature selection method. To defeat the curse of dimensionality, to speed up the classification process, and to get more accurate results a hybrid feature selection model is developed which is a combination of multiple filter methods and a wrapper method. In this model, we employed two sets of Filter Methods-Basic filter methods, a correlation filter method and two Statistical & Ranking filter methods (ANOVA and ROC-AUC) to generate two different subsets of important features, and a Wrapper method (Recursive Feature Elimination with Cross-Validation) is applied on the combined subset to generate a final subset of important features for better prediction results. Five machine learning algorithms-Logistic Regression (LR), Decision Tree, Random Forest (RF), Support Vector Machine (SVM), and K-Nearest Neighbour are used to evaluate the classification accuracy. The proposed hybrid method is applied over four low and four microarray datasets. Outputs are compared to find out which algorithm works best with the proposed model as the results diverge with the machine learning algorithm. Precision, Sensitivity, and Specificity are calculated for each outcome and they demonstrate that the suggested method improved the accuracy of the algorithms.