Uncertainty-Penalized Quadratic Surrogate Optimization Using Cross-Validation-Based Ridge Regression

Abstract

Response surface methodology (RSM) is widely employed for experimental optimization due to its ability to provide a transparent quadratic surrogate constructed from a small, designed experiment, which is often suitable for laboratory and pilot studies. However, the standard approach of fitting a quadratic model by ordinary least squares and optimizing the resulting surface may produce unstable recommendations when the design is ill-conditioned, measurements are noisy, or the response is misspecified. In this study, we introduce an uncertainty-penalized quadratic RSM framework that preserves the experimental design pipeline while enhancing recommendation reliability. The surrogate model is estimated using ridge regression, with the regularization parameter determined by leave-one-out cross validation. A closed-form posterior covariance for the coefficients is derived from a Bayesian interpretation of ridge regression, leading to an analytic predictive variance and an uncertainty estimate across the design region. The recommendation is obtained by minimizing a conservative objective that combines the predicted mean with a tunable multiple of the predictive standard deviation. Five representative scenarios, including correct specification, mild and strong misspecification, severe ill conditioning, and higher-dimensional designs, demonstrate consistent reductions in regret and tail risk metrics, such as the 95th percentile regret and conditional value at risk, compared to ordinary least squares.