MCTS-NC: A thorough GPU parallelization of Monte Carlo Tree Search implemented in Python via numba.cuda

With CUDA computational model in mind, we introduce MCTS-NC (Monte Carlo Tree Search–numba.cuda ). It contains four, fast-operating and thoroughly parallel, variants of the MCTS algorithm. The design of MCTS-NC combines three parallelization levels (leaf / root / tree parallelizations). Additionally, all algorithmic stages – selections, expansions, playouts, backups – employ multiple GPU threads. We apply suitable reduction patterns to carry out summations or max / argmax operations. The implementation uses very few device-host memory transfers, no atomic operations (is lock-free), and takes advantage of threads cooperation. In the mathematical part of this article, we demonstrate how the confidence bounds on estimated action values become tightened by both the number of independent concurrent playouts and the number of independent concurrent trees. The experimental part reports the performance of MCTS-NC on two game examples: Connect4 and Gomoku. All computational results can be exactly reproduced.