Bayesian Optimization Methods for Nonlinear Model Calibration

This work develops and compares seven Gaussian process Bayesian optimization (GPBO) methods for calibrating nonlinear models. We demonstrate through ten (non)linear parameter estimation examples that new BO methods using GP emulators of (computationally expensive) models accurately recovered parameters in 67% of the benchmarking instances compared to 28% for standard GPBO, which uses GP models for the loss objective. When considering noisy or stochastic expensive models, emulator GPBO finds the true parameters in 62% of the instances compared to approximately 0% for gradient-based nonlinear least-squares. We show that GPBO is more efficient than other popular derivative-free search algorithms, including genetic algorithms, the Nelder–Mead algorithm, or the simplicial homology global optimization algorithm. We recommend emulator GPBO with either an expected improvement Monte Carlo approximation or an expected value of the sum of squared errors acquisition function. Finally, we discuss future opportunities to improve these methods and consider applications to expensive stochastic models (e.g., molecular simulations).