Debiasing In-Sample Policy Performance for Small-Data, Large-Scale Optimization

In many modern large-scale decision-making problems, data can be scarce. As a result, traditional methods such as cross-validation perform poorly in evaluating the performance of decision-making policies. In “Debiasing In-Sample Policy Performance for Small-Data, Large-Scale Optimization,” Gupta, Huang, and Rusmevichientong propose a novel estimator of the out-of-sample performance for a policy in data-driven optimization. Unlike cross-validation, their approach avoids sacrificing training data for evaluation. As a result, they theoretically show the estimator is asymptotically unbiased as the problem size grows. Furthermore, they show that the estimator is asymptotically optimal when applied to more specialized “weakly coupled” optimization problems. Finally, using a case study on dispatching emergency medical response services, they demonstrate their proposed method provides more accurate estimates of out-of-sample performance and selects better policies.