MCGA: An effective method for enhancing multi-agent coordination to boost performance in the real-time strategy game

Abstract

Multi-agent reinforcement learning (MARL) excels in addressing complex problems, especially in real-time strategy (RTS) games. While Centralized Training with Decentralized Execution (CTDE) is a popular MARL paradigm, its lack of global guidance during decentralized execution hinders effective coordination, impairing policy learning. The study aims to develop an effective MARL algorithm for application in RTS games, addressing the poor coordination problem in previous CTDE algorithms and thereby improving the algorithm’s performance. To this end, we propose an approach called Improving Multi-Agent Coordination Based on Global-Situation Awareness (MCGA) to facilitate agent coordination by constructing a global-situation awareness estimator that translates the local history into valuable global insights to assist agent decision-making. During training, we introduce an alignment loss and use gradient information from credit assignment to jointly train the global-situation awareness estimator. We conduct extensive experiments in the StarCraft Multi-Agent Challenge (SMAC) game environment. Meanwhile, to further validate the effectiveness of the algorithm, we also conduct experimental validation in the commonly used multi-agent reinforcement learning benchmark environments, Level-Based Foraging (LBF) and Multi-Agent Particle Environment (MPE). The results show that our approach significantly improves the performance of agents in the game and outperforms the state-of-the-art MARL algorithm.