Informed Monte Carlo Tree Search for Real-Time Strategy games

Abstract

The recent success of AlphaGO has shown that it is possible to combine machine learning with Monte Carlo Tree Search (MCTS) in order to improve performance in games with large branching factors. This paper explores the question of whether similar ideas can be applied to a genre of games with an even larger branching factor: Real-Time Strategy games. Specifically, this paper studies (1) the use of Bayesian models to estimate the probability distribution of actions played by a strong player, (2) the incorporation of such models into NaiveMCTS, a MCTS algorithm designed for games with combinatorial branching factors. We call this approach informed MCTS, since it exploits prior information about the game in the form of a probability distribution of actions. We evaluate its performance in the μRTS game simulator, significantly outperforming the previous state of the art.