APPLICATION OF A MIXED INTEGER NONLINEAR PROGRAMMING APPROACH TO VARIABLE SELECTION IN LOGISTIC REGRESSION

Abstract

Variable selection is the process of finding variables relevant to a given dataset in model construction. One of the techniques for variable selection is exponentially evaluating many models with a goodness-of-fit (GOF) measure, for example, Akaike information criterion (AIC). The model with the lowest GOF value is considered as the best model. We proposed a mixed integer nonlinear programming approach to AIC minimization for linear regression and showed that the approach outperformed existing approaches in terms of computational time [13]. In this study, we apply the approach in [13] to AIC minimization for logistic regression and explain that a few of the techniques developed previously [13], for example, relaxation and a branching rule, can be used for the AIC minimization. The proposed approach requires solving relaxation problems, which are unconstrained convex problems. We apply an iterative method with an effective initial guess to solve these problems. We implement the proposed approach via SCIP, which is a noncommercial optimization software and a branch-and-bound framework. We compare the proposed approach with a piecewise linear approximation approach developed by Sato and others [16]. The results of computational experiments show that the proposed approach finds the model with the lowest AIC value if the number of candidates for variables is 45 or lower.