DCB-VIM: An ensemble learning based filter method for feature selection with imbalanced class distribution

Abstract

Feature selection aims to improve predictive performance and interpretability in the analysis of datasets with high dimensional feature spaces. Imbalanced class distribution can make the process of feature selection more severe. Robust methodologies are essential for dealing with this case. Therefore, we present a filter method based on ensemble learning, in which each classifier is built on randomly selected subspaces of features. Variable importance measure is computed based on a class-wise procedure within each classifier, and a feature weighting procedure is subsequently applied. The performance of classifiers is considered in the combination phase of the ensemble learning. Different choices of hyperparameters consisting of the subspace size and the number of classification trees are investigated through simulation studies for determining their effects on the predictive performance. The efficiency of the proposed method is evaluated with respect to predictive performance by different selection strategies based on real data analysis in the presence of class imbalance.