Multi-objective Full Model Selection in temporal databases: Optimizing time and performance

Abstract

In this paper, a multi-objective approach to address the Full Model Selection (FMS) problem in temporal databases is presented. FMS problem is defined as a multi-objective problem where Cross Validation Error Rate (CVER) and the spent runtime (RT) of each candidate model are considered as objectives to be optimized through the well-known NSGA-II algorithm. The intuitive idea is to select competitive models that are not too computationally expensive. Four strategies for preferences handling from the Pareto front obtained by the proposed approach called N2FMS (NSGA-II for FMS) are compared, where three of them are ensemble solutions and the last one is the selection of the nearest solution to a reference point, which results to be the best strategy. The overall assessment suggests that N2FMS is an useful tool to find competitive models and it is capable of suggesting solutions with a lower runtime and competitive error rate.