The Power and Limits of Predictive Approaches to Observational Data-Driven Optimization: The Case of Pricing

We consider data-driven decision making in which data on historical decisions and outcomes are endogenous and lack the necessary features for causal identification (e.g., unconfoundedness or instruments), focusing on data-driven pricing. We study approaches that, for lack of better alternative, optimize the prediction of objective (revenue) given decision (price). Whereas data-driven decision making is transforming modern operations, most large-scale data are observational, with which confounding is inevitable and the strong assumptions necessary for causal identification are dubious. Nonetheless, the inevitable statistical biases may be irrelevant if impact on downstream optimization performance is limited. This paper seeks to formalize and empirically study this question. First, to study the power of decision making with confounded data, by leveraging a special optimization structure, we develop bounds on the suboptimality of pricing using the (noncausal) prediction of historical demand given price. Second, to study the limits of decision making with confounded data, we develop a new hypothesis test for optimality with respect to the true average causal effect on the objective and apply it to interest rate–setting data to assesses whether performance can be distinguished from optimal to statistical significance in practice. Our empirical study demonstrates that predictive approaches can generally be powerful in practice with some limitations.