Encoding Topology Information for Deep Reinforcement Learning with Continuous Action Space

Abstract

In the context of reinforcement learning, the training efficiency can decay exponentially with the size of the state space. Therefore, designing easily-optimized state space representation has remained an open problem. In this paper, we focus on a general and challenging scenario, i.e. reinforcement learning with continuous action spaces. We propose a new representation framework by explicitly encoding topology information such as the geometrical and the kinematic relations among different parts of the agent to make the representation more informative, which results in effective optimization. Extensive experiments were conducted on three settings to demonstrate that our method can remarkably stabilize and speed up the training process.