结合RAVE启发式的改进蒙特卡罗POMDPs在线规划算法

2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS) Pub Date : 2015-11-30 DOI:10.1109/ICSESS.2015.7339109

Peigen Liu, J. Chen, Hongfu Liu

{"title":"结合RAVE启发式的改进蒙特卡罗POMDPs在线规划算法","authors":"Peigen Liu, J. Chen, Hongfu Liu","doi":"10.1109/ICSESS.2015.7339109","DOIUrl":null,"url":null,"abstract":"Partially observable Markov decision processes (POMDPs) provide a kind of general model that can deal with problems in uncertain environment efficiently. There are many different planning methods for POMDPs model. Partially Observable Monte Carlo planning (POMCP) method, using Monte Carlo Tree Search (MCTS) method, which can help break the curse of dimensionality and the curse of history. However, the method has strong dependence on the count of simulations. The POMCP algorithm was improved in this paper by combining Rapid Action Value Estimate (RAVE) method and MCTS. There's less dependence on the count of simulations and higher efficiency in the improved algorithm, which is a promising online planning algorithm. Experimental results on the benchmark problems indicate that efficiency of the improved algorithm is higher than the basic POMCP algorithm.","PeriodicalId":335871,"journal":{"name":"2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS)","volume":"40 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2015-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":"{\"title\":\"An improved Monte Carlo POMDPs online planning algorithm combined with RAVE heuristic\",\"authors\":\"Peigen Liu, J. Chen, Hongfu Liu\",\"doi\":\"10.1109/ICSESS.2015.7339109\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Partially observable Markov decision processes (POMDPs) provide a kind of general model that can deal with problems in uncertain environment efficiently. There are many different planning methods for POMDPs model. Partially Observable Monte Carlo planning (POMCP) method, using Monte Carlo Tree Search (MCTS) method, which can help break the curse of dimensionality and the curse of history. However, the method has strong dependence on the count of simulations. The POMCP algorithm was improved in this paper by combining Rapid Action Value Estimate (RAVE) method and MCTS. There's less dependence on the count of simulations and higher efficiency in the improved algorithm, which is a promising online planning algorithm. Experimental results on the benchmark problems indicate that efficiency of the improved algorithm is higher than the basic POMCP algorithm.\",\"PeriodicalId\":335871,\"journal\":{\"name\":\"2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS)\",\"volume\":\"40 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2015-11-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"2\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICSESS.2015.7339109\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICSESS.2015.7339109","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 2

摘要

部分可观察马尔可夫决策过程(pomdp)提供了一种能够有效处理不确定环境问题的通用模型。pomdp模型有许多不同的规划方法。部分可观察蒙特卡罗规划(POMCP)方法，采用蒙特卡罗树搜索(MCTS)方法，可以帮助打破维数诅咒和历史诅咒。然而，该方法对模拟次数有很强的依赖性。本文将快速动作值估计(Rapid Action Value Estimate, RAVE)方法与MCTS方法相结合，对POMCP算法进行了改进。改进算法对仿真次数的依赖较小，效率较高，是一种很有前途的在线规划算法。在基准问题上的实验结果表明，改进算法的效率高于基本的POMCP算法。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

An improved Monte Carlo POMDPs online planning algorithm combined with RAVE heuristic

Partially observable Markov decision processes (POMDPs) provide a kind of general model that can deal with problems in uncertain environment efficiently. There are many different planning methods for POMDPs model. Partially Observable Monte Carlo planning (POMCP) method, using Monte Carlo Tree Search (MCTS) method, which can help break the curse of dimensionality and the curse of history. However, the method has strong dependence on the count of simulations. The POMCP algorithm was improved in this paper by combining Rapid Action Value Estimate (RAVE) method and MCTS. There's less dependence on the count of simulations and higher efficiency in the improved algorithm, which is a promising online planning algorithm. Experimental results on the benchmark problems indicate that efficiency of the improved algorithm is higher than the basic POMCP algorithm.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS)

自引率

0.00%

发文量