Optimising Feature Selection: A Comparative Study of mRMR-Boruta/RFE Hybrid Approach

Feature selection is an essential component in the data preprocessing pipeline, particularly when dealing with datasets that possess a vast array of dimensions. In this paper, we present a time efficient wrapper technique Boruta to improve the overall complexity of our feature selection process. We have combined this wrapper technique with the filter class Minimum Redundancy Maximum Relevance (mRMR) to enhance the selection of relevant features. Additionally, our scope includes refining a previously proposed hybrid model that combines filter class Minimum Redundancy Maximum Relevance (mRMR) known for faster processing speed with wrapper class Recursive Feature Elimination (RFE) known for its high classification accuracy. We demonstrated the effectiveness of our approach on a variety of datasets and showed that our model is able to identify a smaller and more interpretable subset of features while generating better performance. Our results suggest that the combination of preprocessing and hybrid feature selection model is a promising approach to process a dataset with high dimensions.