A Novel Recursive Ensemble Feature Selection Framework for High-Dimensional Data

Abstract

Ensemble feature selection combines feature subsets with diversity, potentially providing a better approximation of the optimal feature subset. While extensive research has focused on enhancing diversity among ensemble members, its critical role during the aggregation process remains underexplored. To address this gap, we propose a novel Recursive Ensemble Feature Selection (REFS) framework that explicitly incorporates diversity into the aggregation phase to improve both robustness and accuracy. The framework comprises three key components: 1) a randomization-based feature mapping strategy (RS) to generate diverse base feature selectors optimized for performance; 2) a quantitative diversity metric (DM) to evaluate the complementarity of these selectors; and 3) a fuzzy aggregation (FA) method that leverages order statistics, rank scores, and weight information to effectively integrate multiple ranked feature lists. Experimental evaluations on fifteen real-world datasets demonstrate that REFS consistently outperforms competitive methods in terms of classification accuracy and resilience to parameter variations. By explicitly integrating diversity into the aggregation process, REFS provides a more comprehensive and effective approach to feature selection, paving the way for improved predictive performance across diverse applications.