时间平均MDP中的最优策略是什么?

Q4 Computer Science

Performance Evaluation Review Pub Date : 2023-09-28 DOI:10.1145/3626570.3626582

Nicolas Gast, Bruno Gaujal, Kimang Khun

{"title":"时间平均MDP中的最优策略是什么?","authors":"Nicolas Gast, Bruno Gaujal, Kimang Khun","doi":"10.1145/3626570.3626582","DOIUrl":null,"url":null,"abstract":"This paper discusses the notion of optimality for time-average MDPs. We argue that while most authors claim to use the \"average reward\" criteria, the notion that is implicitly used is in fact the notion of what we call Bellman optimality. We show that it does not coincide with other existing notions of optimality, like gain-optimality and bias-optimality but has strong connection with canonical-policies (policies that are optimal for any finite horizons) as well as value iteration and policy iterations algorithms.","PeriodicalId":35745,"journal":{"name":"Performance Evaluation Review","volume":"25 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"What is an Optimal Policy in Time-Average MDP?\",\"authors\":\"Nicolas Gast, Bruno Gaujal, Kimang Khun\",\"doi\":\"10.1145/3626570.3626582\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"This paper discusses the notion of optimality for time-average MDPs. We argue that while most authors claim to use the \\\"average reward\\\" criteria, the notion that is implicitly used is in fact the notion of what we call Bellman optimality. We show that it does not coincide with other existing notions of optimality, like gain-optimality and bias-optimality but has strong connection with canonical-policies (policies that are optimal for any finite horizons) as well as value iteration and policy iterations algorithms.\",\"PeriodicalId\":35745,\"journal\":{\"name\":\"Performance Evaluation Review\",\"volume\":\"25 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2023-09-28\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Performance Evaluation Review\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3626570.3626582\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"Computer Science\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Performance Evaluation Review","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3626570.3626582","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"Computer Science","Score":null,"Total":0}

引用次数: 0

摘要

本文讨论了时间平均mdp的最优性概念。我们认为，虽然大多数作者声称使用“平均奖励”标准，但隐含使用的概念实际上是我们称之为Bellman最优性的概念。我们表明，它与其他现有的最优性概念不一致，如增益最优性和偏差最优性，但与经典策略(对任何有限视界最优的策略)以及值迭代和策略迭代算法有很强的联系。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

What is an Optimal Policy in Time-Average MDP?

This paper discusses the notion of optimality for time-average MDPs. We argue that while most authors claim to use the "average reward" criteria, the notion that is implicitly used is in fact the notion of what we call Bellman optimality. We show that it does not coincide with other existing notions of optimality, like gain-optimality and bias-optimality but has strong connection with canonical-policies (policies that are optimal for any finite horizons) as well as value iteration and policy iterations algorithms.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Performance Evaluation Review Computer Science-Computer Networks and Communications

CiteScore

1.00

自引率

0.00%

发文量

193