DRLeague: a Novel 3D Environment for Training Reinforcement Learning Agents

Abstract

The development of autonomous agents performing unique interactions that resemble human-like behavior is currently driven by Deep Reinforcement Learning (DRL) techniques combined with complex virtual environments. These constitute an active field of research that is fueled by environments usually inspired or borrowed from video games. Although works in the area commonly do not make use of trending 3D games, these games are interesting testbeds for more complex and compelling behaviors, as they tend to explore more variables than their predecessors. This paper introduces DRLeague, a novel DRL environment, proposed to be open-source, and easily customizable, which supports mechanics for 3D games inspired by the popular “car football” game Rocket League. Besides the typical gameplay, we implemented four challenging minigames based on the mechanics from this title with advanced physics simulation and fine-grained car control: penalty shoot, multiplayer penalty shoot, barrier kick, and aerial shoot, each of these requiring more complex skills than the previous ones. Finally, we provide solid baseline experimental results showing the learning progress of agents using Unity's ML-Agents toolkit, evidencing DRLeague as a suitable testbed in the application of machine learning techniques.