Supervised Rank Aggregation (SRA): A Novel Rank Aggregation Approach for Ensemble-based Feature Selection

Abstract

Feature selection (FS) is critical for high dimensional data analysis. Ensemble based feature selection (EFS) is a commonly used approach to develop FS techniques. Rank aggregation (RA) is an essential step in EFS where results from multiple models are pooled to estimate feature importance. However, the literature primarily relies on static rule-based methods to perform this step which may not always provide an optimal feature set. The objective of this study is to improve the EFS performance using dynamic learning in RA step. This study proposes a novel Supervised Rank Aggregation (SRA) approach to allow RA step to dynamically learn and adapt the model aggregation rules to obtain feature importance.Method: This study proposes a novel Supervised Rank Aggregation (SRA) approach to allow RA step to dynamically learn and adapt the model aggregation rules to obtain feature importance. We evaluate the performance of the algorithm using simulation studies and implement it into real research studies, and compare its performance with various existing RA methods. The proposed SRA method provides better or at par performance in terms of feature selection and predictive performance of the model compared to existing methods. SRA method provides an alternative to the existing approaches of RA for EFS. While the current study is limited to the continuous cross-sectional outcome, other endpoints such as longitudinal, categorical, and time-to-event data could also be used.