A Local-branching Heuristic for the Best Subset Selection Problem in Linear Regression

Abstract

The best subset selection problem in linear regression consists of selecting a small subset with a given maximum cardinality of a set of features, i.e explanatory variables, to build a linear regression model that is able to explain a given set of observations of a response variable as exactly as possible. The motivation in building linear regression models that include only a small number of features is that these models are easier to interpret. In this paper, we present a heuristic based on the concept of local branching. Such a heuristic repeatedly performs local-search iterations by applying mixed-integer programming. In each local-search iteration, we consider a different randomly selected subset of the features to reduce the required computational time. The results of our computational tests demonstrate that the proposed local-branching heuristic delivers better linear regression models than a pure mixed-integer programming approach within a limited amount of computational time.