Hierarchical Monte Carlo Tree Search for Latent Skill Planning

Abstract

Monte Carlo Tree Search (MCTS) continues to confront the issue of exponential complexity growth in certain tasks when the planning horizon is excessively long, causing the trajectory’s past to grow exponentially. Our study presents Hierarchical MCTS Latent Skill Planner, an algorithm based on skill discovery that automatically identifies skills based on intrinsic rewards and integrates them with MCTS, enabling efficient decision-making at a higher level. In the grid world maze domain, we found that latent skill search outperformed the standard MCTS approach that do not contain skills in terms of efficiency and performance.