Time Management for Monte Carlo Tree Search

Q2 Computer Science

IEEE Transactions on Computational Intelligence and AI in Games Pub Date : 2016-09-01 DOI:10.1109/TCIAIG.2015.2443123

Hendrik Baier, M. Winands

{"title":"Time Management for Monte Carlo Tree Search","authors":"Hendrik Baier, M. Winands","doi":"10.1109/TCIAIG.2015.2443123","DOIUrl":null,"url":null,"abstract":"Monte Carlo Tree Search (MCTS) is a popular approach for tree search in a variety of games. While MCTS allows for fine-grained time control, not much has been published on time management for MCTS programs under tournament conditions. This paper first investigates the effects of various time-management strategies on playing strength in the challenging game of Go. A number of domain-independent strategies are then tested in the domains Connect-4, Breakthrough, Othello, and Catch the Lion. We consider strategies taken from the literature as well as newly proposed and improved ones. Strategies include both semi-dynamic strategies that decide about time allocation for each search before it is started, and dynamic strategies that influence the duration of each move search while it is already running. Furthermore, we analyze the effects of time management strategies on the distribution of time over the moves of an average game, allowing us to partly explain their performance. In the experiments, the domain-independent strategy STOP provides a significant improvement over the state of the art in Go, and is the most effective time management strategy tested in all five domains.","PeriodicalId":49192,"journal":{"name":"IEEE Transactions on Computational Intelligence and AI in Games","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2016-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TCIAIG.2015.2443123","citationCount":"7","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Computational Intelligence and AI in Games","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/TCIAIG.2015.2443123","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Computer Science","Score":null,"Total":0}

引用次数: 7

Abstract

Monte Carlo Tree Search (MCTS) is a popular approach for tree search in a variety of games. While MCTS allows for fine-grained time control, not much has been published on time management for MCTS programs under tournament conditions. This paper first investigates the effects of various time-management strategies on playing strength in the challenging game of Go. A number of domain-independent strategies are then tested in the domains Connect-4, Breakthrough, Othello, and Catch the Lion. We consider strategies taken from the literature as well as newly proposed and improved ones. Strategies include both semi-dynamic strategies that decide about time allocation for each search before it is started, and dynamic strategies that influence the duration of each move search while it is already running. Furthermore, we analyze the effects of time management strategies on the distribution of time over the moves of an average game, allowing us to partly explain their performance. In the experiments, the domain-independent strategy STOP provides a significant improvement over the state of the art in Go, and is the most effective time management strategy tested in all five domains.

查看原文本刊更多论文

蒙特卡洛树搜索的时间管理

蒙特卡罗树搜索(MCTS)是一种在各种游戏中流行的树搜索方法。虽然MCTS允许细粒度的时间控制，但关于比赛条件下MCTS程序的时间管理的文章还不多。本文首先研究了不同时间管理策略对围棋棋力的影响。然后，在Connect-4、Breakthrough、Othello和Catch the Lion领域中测试了许多与领域无关的策略。我们考虑从文献中采取的策略以及新提出的和改进的策略。策略既包括半动态策略(在每次搜索开始前决定其时间分配)，也包括动态策略(在每次移动搜索已经运行时影响其持续时间)。此外，我们分析了时间管理策略对平均游戏中移动时间分配的影响，使我们能够部分解释它们的表现。在实验中，与领域无关的STOP策略提供了对围棋最新状态的显著改进，并且是在所有五个领域中测试过的最有效的时间管理策略。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Computational Intelligence and AI in Games COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE-COMPUTER SCIENCE, SOFTWARE ENGINEERING

CiteScore

4.60

自引率

0.00%

发文量

审稿时长

>12 weeks

期刊介绍： Cessation. The IEEE Transactions on Computational Intelligence and AI in Games (T-CIAIG) publishes archival journal quality original papers in computational intelligence and related areas in artificial intelligence applied to games, including but not limited to videogames, mathematical games, human–computer interactions in games, and games involving physical objects. Emphasis is placed on the use of these methods to improve performance in and understanding of the dynamics of games, as well as gaining insight into the properties of the methods as applied to games. It also includes using games as a platform for building intelligent embedded agents for the real world. Papers connecting games to all areas of computational intelligence and traditional AI are considered.