On the Continuity of the Projection Mapping from Strategic Measures to Occupation Measures in Absorbing Markov Decision Processes

IF 1.6 2区数学 Q2 MATHEMATICS, APPLIED

Applied Mathematics and Optimization Pub Date : 2024-04-12 DOI:10.1007/s00245-024-10124-7

Alexey Piunovskiy, Yi Zhang

引用次数: 0

Abstract

In this paper, we prove the following assertion for an absorbing Markov decision process (MDP) with the given initial distribution, which is also assumed to be semi-continuous: the continuity of the projection mapping from the space of strategic measures to the space of occupation measures, both endowed with their weak topologies, is equivalent to the MDP model being uniformly absorbing. An example demonstrates, among other interesting scenarios, that for an absorbing (but not uniformly absorbing) semi-continuous MDP with the given initial distribution, the space of occupation measures can fail to be compact in the weak topology.

Abstract Image

查看原文本刊更多论文

论吸收马尔可夫决策过程中从战略措施到职业措施的投影映射的连续性

本文针对具有给定初始分布的吸收马尔可夫决策过程（MDP）证明了以下论断：从策略度量空间到占领度量空间（两者都具有弱拓扑）的投影映射的连续性等同于 MDP 模型具有均匀吸收性。一个例子表明，对于具有给定初始分布的吸收性（但非均匀吸收性）半连续 MDP 模型，其占领度量空间在弱拓扑中可能不紧凑。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Applied Mathematics and Optimization 数学-应用数学

CiteScore

3.30

自引率

5.60%

发文量

103

审稿时长

>12 weeks

期刊介绍： The Applied Mathematics and Optimization Journal covers a broad range of mathematical methods in particular those that bridge with optimization and have some connection with applications. Core topics include calculus of variations, partial differential equations, stochastic control, optimization of deterministic or stochastic systems in discrete or continuous time, homogenization, control theory, mean field games, dynamic games and optimal transport. Algorithmic, data analytic, machine learning and numerical methods which support the modeling and analysis of optimization problems are encouraged. Of great interest are papers which show some novel idea in either the theory or model which include some connection with potential applications in science and engineering.