Reinforcement Learning Algorithms Performance Comparison on the Game of DeepRTS

Abstract

In this paper, i) we build a framework using macro actions on top of DeepRTS, which is specifically developed for learning purposes, and ii) we train a number of deep reinforcement learning based agents and then we conduct a detailed performance analysis on the performance of these agents. We make changes on the publicly available version of DeepRTS to make it parallel to real life planning problems and make it trainable with a decent hardware. Firstly, we create a set of macro actions based on human heuristics and we train our agent using only these actions. Secondly, similar to the real life planning problem, we change the problem such that we simultaneously take actions for all available unit at a time step. When we consider this situation together with duration of macro actions being different, multiple actions may start and multiple actions ends at any time step. Therefore, the problem draw apart from classical reinforcement learning problem in a certain amount. In addition, the credit assignment problem becomes more sophisticated since each action lasts a duration and at any time multiple actions are conducted. We create three rule based agents to use them as enemy players in training of our agents. We report the performance of the agents as player-1 and player-2 against each others.