Monte Carlo Tree Search with macro-actions and heuristic route planning for the Physical Travelling Salesman Problem

Abstract

We present a controller for the Physical Travelling Salesman Problem (PTSP), a path planning and steering problem in a simulated continuous real-time domain. Our approach is hierarchical, using domain-specific algorithms and heuristics to plan a coarse-grained route and Monte Carlo Tree Search (MCTS) to plan and steer along fine-grained paths. The MCTS component uses macro-actions to decrease the number of decisions to be made per unit of time and thus drastically reduce the size of the decision tree. Results from the 2012 WCCI PTSP Competition show that this approach significantly and consistently outperforms all other submitted AI controllers, and is competitive with strong human players. Our approach has potential applications to many other problems in movement planning and control, including video games.