A Markov Decision Model for Managing Display-Advertising Campaigns

Abstract

Problem definition: Managers in ad agencies are responsible for delivering digital ads to viewers on behalf of advertisers, subject to the terms specified in the ad campaigns. They need to develop bidding policies to obtain viewers on an ad exchange and allocate them to the campaigns to maximize the agency’s profits, subject to the goals of the ad campaigns. Academic/practical relevance: Determining a rigorous solution methodology is complicated by uncertainties in the arrival rates of viewers and campaigns, as well as uncertainty in the outcomes of bids on the ad exchange. In practice, ad hoc strategies are often deployed. Our methodology jointly determines optimal bidding and viewer-allocation strategies and obtains insights about the characteristics of the optimal policies. Methodology: New ad campaigns and viewers are treated as Poisson arrivals, and the resulting model is a Markov decision process, where the state of the system is the number of undelivered impressions in queue for each campaign type in each period. We develop solution methods for bid optimization and viewer allocation and perform a sensitivity analysis with respect to the key problem parameters. Results: We solve for the optimal dynamic, state-dependent bidding and allocation policies as a function of the number of ad impressions in queue, for both the finite horizon and steady-state cases. We show that the resulting optimization problems are strictly concave in the decision variables and develop and evaluate a heuristic method that can be applied to large problems. Managerial implications: Numerical analysis of our heuristic solution shows that its errors are generally small and that the optimal dynamic, state-dependent bidding policies obtained by our model are significantly better than optimal static policies. Our proposed approach is managerially attractive because it is easy to implement in practice. We identify the capacity of the impression queue as an important managerial control lever and show that it can be more effective than using higher bids to reduce delay penalties. We quantify potential operational benefits from the consolidation of ad campaigns, as well as merging ad exchanges. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2022.1142 .