Automatic Composite Action Discovery for Hierarchical Reinforcement Learning

Abstract

Even with recent advances in standard reinforcement learning, hierarchical reinforcement learning has been discussed as a promising approach to solve complex problems. From human-designed abstraction, planning or learning with composite actions are well-understood, but without human intervention, producing abstract (or composite) actions automatically is one of the remaining challenges. We separate this action discovery from reinforcement learning problem and investigate on searching impactful composite actions that can make meaningful changes in state space. We discuss the efficiency and flexibility of the suggested model by interpreting and analyzing the discovered composite actions with different deep reinforcement learning algorithms in different environments.