Certifiably optimal direction estimation in sparse single-index model

Abstract

In this paper, we propose a novel method for coefficient estimation in sparse single-index models (SIM). Our approach employs a customized branch-and-bound algorithm to efficiently solve the non-convex problem of sparse direction estimation, which arises from the discrete nature of variable selection. To address this non-convex optimization problem, we derive upper bounds using techniques such as spectral decomposition, matrix inequalities, and the Gershgorin circle theorem, while the lower bounds are obtained through methods like vector truncation and adaptations of the Rifle algorithm. Furthermore, we design customized branching and node selection strategies, with hyperparameters chosen based on AIC, BIC, and HBIC criteria. We prove the convergence of our algorithm, ensuring it reliably reaches optimal solutions. Extensive simulation studies and real data analysis further illustrate the reliable performance and applicability of our proposed method.