Encoding categorical variables in physics-informed graphs for Bayesian Optimization

Abstract

The handling of categorical variables is a major challenge while using Bayesian Optimization (BO). Recent publications address this problem by introducing novel BO-frameworks adjusted specifically for categorical inputs. A promising approach, named "Gryffin", highlights the importance of using prior physical knowledge about categorical variables and shows significant performance improvements in comparison to other state of the art frameworks. However, the runtime of this approach increases significantly when the number of categorical choices is large. Therefore, we propose an alternative method for utilizing prior physical knowledge in BO by using physics-informed graphs. Our approach is based on a BO framework named "Combo", which allows handling graph-structures as input variables. In contrast to handling the categorical variables in complete graphs as originally proposed, we present a method for reshaping and simplifying the graph-structures based on prior physical knowledge. Physical similarities of categorical choices are encoded in graph edges, inducing proximity in the search space and enabling more precise predictions of a surrogate model. The optimization performance improves in comparison to the default Combo approach and other state of the art optimization techniques.