A modified least angle regression algorithm for interaction selection with heredity

Abstract

In many practical problems, the main effects alone may not be enough to capture the relationship between the response and predictors, and the interaction effects are often of interest to scientific researchers. In considering a regression model with main effects and all possible two‐way interaction effects, which we call the two‐way interaction model, there is an important challenge—computational burden. One way to reduce the aforementioned problems is to consider the heredity constraint between the main and interaction effects. The heredity constraint assumes that a given interaction effect is significant only when the corresponding main effects are significant. Various sparse penalized methods to reflect the heredity constraint have been proposed, but those algorithms are still computationally demanding and can be applied to data where the dimension of the main effects is only few hundreds. In this paper, we propose a modification of the LARS algorithm for selecting interaction effects under the heredity constraint, which can be applied to high‐dimensional data. Our numerical studies confirm that the proposed modified LARS algorithm is much faster and spends less memory than its competitors but has comparable prediction accuracies when the dimension of covariates is large.