Building predictive models in two stages with meta-learning templates optimized by genetic programming

Abstract

The model selection stage is one of the most difficult in predictive modeling. To select a model with a highest generalization performance involves benchmarking huge number of candidate models or algorithms. Often, a final model is selected without considering potentially high quality candidates just because there are too many possibilities. Improper benchmarking methodology often leads to biased estimates of model generalization performance. Automation of the model selection stage is possible, however the computational complexity is huge especially when ensembles of models and optimization of input features should be also considered. In this paper we show, how to automate model selection process in a way that allows to search for complex hierarchies of ensemble models while maintaining computational tractability. We introduce two-stage learning, meta-learning templates optimized by evolutionary programming with anytime properties to be able to deliver and maintain data-tailored algorithms and models in a reasonable time without human interaction. Co-evolution if inputs together with optimization of templates enabled to solve algorithm selection problem efficiently for variety of datasets.