Combining variable screening methods for model averaging in high-dimensional data analysis

Abstract

Both model averaging (MA) and variable screening have been subjects of extensive research. The literature has introduced various methods for variable screening, typically relying on a single approach, such as marginal correlation. However, the performances of such methods may be very poor outside of their specifically applicable scenarios, and it is often difficult to know which scenarios are proper in real data analysis. In this study, we propose a method called Recommendation and Trimmed Mean Scoring (RTMS) for variable screening, which enhances reliability and stability of variable screening by integrating rankings from multiple screening methods. We introduce a practical concept of approximate consistency in ranking of variables, establishing a theoretical property of the RTMS method in both hard sparse and gradually decaying coefficient scenarios. Simulation and empirical results demonstrate that the RTMS method outperforms individual ranking methods in terms of variable screening and predictive performance of MA as well.