Limit Action Space to Enhance Drone Control with Deep Reinforcement Learning

Abstract

Although many research progresses on deep reinforcement learning, it is not yet perfect. It may take too much time or even fail to solve the problem. Therefore, simplifying the problem by intentionally limiting the agent’s action space should help train the agent efficiently and effectively. To verify that, in this paper, we analyze the performances of various action space designs for controlling a drone with deep reinforcement learning. We have designed six different action spaces according to the degree of freedom to analyze the effect of limiting the agent’s action space on performance metrics such as travel distance and time, goal rate, and total reward. We show that by limiting the degree of freedom, the agent learns to reach the goal faster with less travel distance and achieve a higher goal rate and reward.