An improved approach to reinforcement learning in Computer Go

Abstract

Monte-Carlo Tree Search (MCTS) has revolutionized, Computer Go, with programs based on the algorithm, achieving a level of play that previously seemed decades away., However, since the technique involves constructing a search tree, its performance tends to degrade in larger state spaces. Dyna-2, is a hybrid approach that attempts to overcome this shortcoming, by combining Monte-Carlo methods with state abstraction. While, not competitive with the strongest MCTS-based programs, the, Dyna-2-based program RLGO achieved the highest ever rating, by a traditional program on the 9×9 Computer Go Server. Plain, Dyna-2 uses _-greedy exploration and a flat learning rate, but we, show that the performance of the algorithm can be significantly, improved by making some relatively minor adjustments to this, configuration. Our strongest modified program achieved an Elo, rating 289 points higher than the original in head-to-head play, equivalent to an expected win rate of 84%.