Robust variable selection criteria for the penalized regression

Abstract

We develop a robust variable selection framework that integrates divergence-based M-estimation with penalization. The proposed method yields regression parameter estimates that are resistant to outliers while simultaneously identifying the most relevant explanatory variables. The asymptotic distribution and influence function of the estimators are derived. Classical model selection criteria such as Mallows’ $C_p$ and the Akaike information criterion (AIC) are known to deteriorate under heavy-tailed errors or contamination. To address this issue, we introduce robust counterparts of these criteria, constructed from our divergence-based estimators. The proposed approach substantially improves variable selection and prediction performance in the presence of outliers, while maintaining competitiveness with state-of-the-art robust high-dimensional methods. The practical utility of the procedure is further demonstrated through an analysis of the plasma Beta-Carotene dataset.