Tacit mechanism: Bridging pre-training of individuality to multi-agent adversarial coordination

Abstract

To tackle the multi-agent adversarial coordination problem, current multi-agent reinforcement learning (MARL) algorithms primarily depend on team-based rewards to update agent policies. However, they do not fully exploit the spatial relationships and their variant trends, thereby limiting overall performance. Inspired by human tactics, we propose the concept of tacit behavior to enhance the efficiency of multi-agent reinforcement learning through the refinement of the learning process. This paper introduces a novel two-phase framework to learn Pre-trained Tacit Behavior for efficient multi-agent adversarial Coordination (PTBC). The framework consists of a tacit pre-training phase and a centralized adversarial training phase. For pre-training the tacit behaviors, we develop a pattern mechanism and a tacit mechanism to integrate spatial relationships among agents, which dynamically guide agents’ actions to gain spatial advantages for coordination. In the subsequent centralized adversarial training phase, we utilize the pre-trained network to enhance the formation of advantageous spatial positioning, achieving more efficient learning performance. Our experimental results in the predator-prey and StarCraft Multi-Agent Challenge (SMAC) environments demonstrate the effectiveness of our method through comparisons with several algorithms exhibiting distinct strengths. Additionally, by visualizing the agents’ performance in adversarial tasks, we validate that incorporating inter-agent relationships enables agents with pre-trained tacit behavior to achieve more advantageous coordination. Extensive ablation studies demonstrate the critical role of tacit guidance and the general applicability of the PTBC framework.