Efficient approaches to interleaved sampling of training data for symbolic regression

Abstract

The ability to generalize beyond the training set is paramount for any machine learning algorithm and Genetic Programming (GP) is no exception. This paper investigates a recently proposed technique to improve generalisation in GP, termed Interleaved Sampling where GP alternates between using the entire data set and only a single data point in alternate generations. This paper proposes two alternatives to using a single data point: the use of random search instead of a single data point, and simply minimising the tree size. Both the approaches are more efficient than the original Interleaved Sampling because they simply do not evaluate the fitness in half the number of generations. The results show that in terms of generalisation, random search and size minimisation are as effective as the original Interleaved Sampling; however, they are computationally more efficient in terms of data processing. Size minimisation is particularly interesting because it completely prevents bloat while still being competitive in terms of training results as well as generalisation. The tree sizes with size minimisation are substantially smaller reducing the computational expense substantially.