使用马尔可夫风险度量的风险敏感马尔可夫决策问题中建模错误的鲁棒性

Shiping Shao;Abhishek Gupta
{"title":"使用马尔可夫风险度量的风险敏感马尔可夫决策问题中建模错误的鲁棒性","authors":"Shiping Shao;Abhishek Gupta","doi":"10.1109/OJCSYS.2025.3538267","DOIUrl":null,"url":null,"abstract":"We consider risk-sensitive Markov decision processes (MDPs), where the MDP model is influenced by a parameter which takes values in a compact metric space. These situations arise when the underlying dynamics of the system depend on parameters that drifts over time. For example, mass of a vehicle depends on the number of passengers in the vehicle, which may change from one trip to another. Similarly, the energy demand of a building depends on the local weather, which changes every hour of the day. We identify sufficient conditions under which small perturbations in the model parameters lead to small changes in the optimal value function and optimal policy. This is achieved by establishing the continuity of the value function with respect to the parameters. A direct consequence of this result is that an optimal policy under a specific parameter remains near-optimal if the parameter is perturbed slightly. Implications of the results for data-driven decision-making, decision-making with preference uncertainty, and systems with changing noise distributions are discussed.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"4 ","pages":"70-82"},"PeriodicalIF":0.0000,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10872799","citationCount":"0","resultStr":"{\"title\":\"Robustness to Modeling Errors in Risk-Sensitive Markov Decision Problems With Markov Risk Measures\",\"authors\":\"Shiping Shao;Abhishek Gupta\",\"doi\":\"10.1109/OJCSYS.2025.3538267\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"We consider risk-sensitive Markov decision processes (MDPs), where the MDP model is influenced by a parameter which takes values in a compact metric space. These situations arise when the underlying dynamics of the system depend on parameters that drifts over time. For example, mass of a vehicle depends on the number of passengers in the vehicle, which may change from one trip to another. Similarly, the energy demand of a building depends on the local weather, which changes every hour of the day. We identify sufficient conditions under which small perturbations in the model parameters lead to small changes in the optimal value function and optimal policy. This is achieved by establishing the continuity of the value function with respect to the parameters. A direct consequence of this result is that an optimal policy under a specific parameter remains near-optimal if the parameter is perturbed slightly. Implications of the results for data-driven decision-making, decision-making with preference uncertainty, and systems with changing noise distributions are discussed.\",\"PeriodicalId\":73299,\"journal\":{\"name\":\"IEEE open journal of control systems\",\"volume\":\"4 \",\"pages\":\"70-82\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2025-02-04\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10872799\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE open journal of control systems\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10872799/\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE open journal of control systems","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10872799/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

我们考虑风险敏感马尔可夫决策过程(MDP),其中MDP模型受到一个参数的影响,该参数在紧致度量空间中取值。当系统的潜在动态依赖于随时间漂移的参数时,就会出现这些情况。例如,车辆的质量取决于车辆中的乘客数量,这可能会在每次旅行中发生变化。同样,建筑物的能源需求也取决于当地的天气,而当地的天气每时每刻都在变化。我们确定了模型参数中的小扰动导致最优值函数和最优策略的小变化的充分条件。这是通过建立值函数相对于参数的连续性来实现的。这个结果的一个直接结果是,在特定参数下的最优策略,如果参数稍微受到扰动,仍然是接近最优的。讨论了结果对数据驱动决策、偏好不确定性决策和噪声分布变化系统的影响。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Robustness to Modeling Errors in Risk-Sensitive Markov Decision Problems With Markov Risk Measures
We consider risk-sensitive Markov decision processes (MDPs), where the MDP model is influenced by a parameter which takes values in a compact metric space. These situations arise when the underlying dynamics of the system depend on parameters that drifts over time. For example, mass of a vehicle depends on the number of passengers in the vehicle, which may change from one trip to another. Similarly, the energy demand of a building depends on the local weather, which changes every hour of the day. We identify sufficient conditions under which small perturbations in the model parameters lead to small changes in the optimal value function and optimal policy. This is achieved by establishing the continuity of the value function with respect to the parameters. A direct consequence of this result is that an optimal policy under a specific parameter remains near-optimal if the parameter is perturbed slightly. Implications of the results for data-driven decision-making, decision-making with preference uncertainty, and systems with changing noise distributions are discussed.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信