Mean-variance optimization in finite horizon Markov decision processes and its application to revenue management

Abstract

In many applications, risk-averse decision-making is crucial. In this context, the mean–variance (MV) criterion is widely accepted and often used to find the right balance between maximizing expected rewards and avoiding poor performances. In dynamic settings, however, it is challenging to efficiently compute policies under the MV objective and hence, surrogates like the exponential utility model are often used. In this paper, we consider MV optimization for discrete time Markov decision processes (MDP) with finite horizon. Our approach is based on a system of tractable subproblems with distorted variance that allows to identify mean–variance combinations that cannot be attained. The number of subproblems to solve can be chosen such that a predetermined ex-ante optimality gap is obtained. We illustrate the effectiveness and the applicability of our approach for different revenue management examples. We find that competitive ex-ante and ex-post optimality gaps lower than 0.0001% can be reliably obtained with acceptable computational effort.