Risk-Averse Markov Decision Processes Through a Distributional Lens

IF 1.4 3区数学 Q2 MATHEMATICS, APPLIED

Mathematics of Operations Research Pub Date : 2024-07-17 DOI:10.1287/moor.2023.0211

Ziteng Cheng, Sebastian Jaimungal

引用次数: 0

Abstract

By adopting a distributional viewpoint on law-invariant convex risk measures, we construct dynamic risk measures (DRMs) at the distributional level. We then apply these DRMs to investigate Markov decision processes, incorporating latent costs, random actions, and weakly continuous transition kernels. Furthermore, the proposed DRMs allow risk aversion to change dynamically. Under mild assumptions, we derive a dynamic programming principle and show the existence of an optimal policy in both finite and infinite time horizons. Moreover, we provide a sufficient condition for the optimality of deterministic actions. For illustration, we conclude the paper with examples from optimal liquidation with limit order books and autonomous driving.Funding: This work was supported by Natural Sciences and Engineering Research Council of Canada [Grants RGPAS-2018-522715 and RGPIN-2018-05705].

查看原文本刊更多论文

从分布角度看风险厌恶马尔可夫决策过程

通过采用分布观点来看待不变法凸风险度量，我们在分布层面上构建了动态风险度量（DRMs）。然后，我们将这些 DRMs 应用于研究马尔可夫决策过程，其中包含了潜在成本、随机行动和弱连续的过渡核。此外，所提出的 DRM 允许风险规避发生动态变化。在温和的假设条件下，我们推导出了动态编程原理，并证明在有限和无限时间跨度内都存在最优政策。此外，我们还为确定性行动的最优性提供了充分条件。最后，我们以限价订单簿和自动驾驶的最优清算为例进行了说明：这项工作得到了加拿大自然科学与工程研究理事会 [RGPAS-2018-522715 和 RGPIN-2018-05705] 的支持。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Mathematics of Operations Research 管理科学-应用数学

CiteScore

3.40

自引率

5.90%

发文量

178

审稿时长

15.0 months

期刊介绍： Mathematics of Operations Research is an international journal of the Institute for Operations Research and the Management Sciences (INFORMS). The journal invites articles concerned with the mathematical and computational foundations in the areas of continuous, discrete, and stochastic optimization; mathematical programming; dynamic programming; stochastic processes; stochastic models; simulation methodology; control and adaptation; networks; game theory; and decision theory. Also sought are contributions to learning theory and machine learning that have special relevance to decision making, operations research, and management science. The emphasis is on originality, quality, and importance; correctness alone is not sufficient. Significant developments in operations research and management science not having substantial mathematical interest should be directed to other journals such as Management Science or Operations Research.