Transformation-Interaction-Rational Representation for Symbolic Regression: A Detailed Analysis of SRBench Results

Abstract

Symbolic Regression searches for a parametric model with the optimal value of the parameters that best fits a set of samples to a measured target. The desired solution has a balance between accuracy and interpretability. Commonly, there is no constraint in the way the functions are composed in the expression or where the numerical parameters are placed, which can potentially lead to expressions that require a nonlinear optimization to find the optimal parameters. The representation called Interaction-Transformation alleviates this problem by describing expressions as a linear regression of the composition of functions applied to the interaction of the variables. One advantage is that any model that follows this representation is linear in its parameters, allowing an efficient computation. More recently, this representation was extended by applying a univariate function to the rational function of two Interaction-Transformation expressions, called Transformation-Interaction-Rational (TIR). The use of this representation was shown to be competitive with the current literature of Symbolic Regression. In this article, we make a detailed analysis of these results using the SRBench benchmark. For this purpose, we split the datasets into different categories to understand the algorithm behavior in different settings. We also test the use of nonlinear optimization to adjust the numerical parameters instead of Ordinary Least Squares. We find through the experiments that TIR has some difficulties handling high-dimensional and noisy datasets, especially when most of the variables are composed of random noise. These results point to new directions for improving the evolutionary search of TIR expressions.