Generalized linear tree: a flexible algorithm for predicting continuous variables

Abstract

Tree-based models are popular among regression methods to predict continuous variables. Also, Generalized Linear Models (GLMs) are pretty standard in many statistical applications and provide a generalization to many of the most commonly applied statistical procedures. However, in most regression tree methods, there is only one theoretical model associated for prediction in the final nodes, like multiple linear regression, logistic regressions, polynomial models, Poisson models, among others. We, therefore, propose a new tree method in which we estimate a GLM in each leaf node of the estimated tree including variable selection, new hyperparameters optimization, and tree pruning. Our method, called Generalized linear tree (GLT), has shown to be competitive compared to other well-known regression methods in real datasets, with the advantages and estimation flexibility provided by GLMs.