Refining Co-operative Competition of Robocup Soccer with Reinforcement Learning

Abstract

Reinforcement learning (RL) has been widely applied in RoboCup soccer games because of its great potential in enhancing the performance for the model-free competitive scenarios. In recent years, researchers have made a lot of efforts on reducing the input size of RL in order to speed up the training process of the RoboCup soccer agents. In this work, we proposed an improved DQN algorithm named Hierarchical Movement Grouped Deep-Q-Network (HMG-DQN). The algorithm can be trained when actions are in high hierarchy of movement groups, especially in the co-operative competition scenarios, such as 2v1 and 3v2 break-throughs. We conducted the experiments on a simulation platform based on RoboCup SPL rules, and the results showed that our improved algorithm has significantly improved the winning rate compared with DQN.