Efficient multi-agent communication via entity-aware causal network

Abstract

Communication is considered as a crucial approach for solving complicated multi-agent reinforcement learning (MARL) cooperative tasks. However, existing approaches rely on predefined agent orders and identifiers to learn targeted communication. The predefined approaches ignore the prior knowledge that the selection of communication targets is solely related to agents’ states rather than their orders or identifiers, which leads to poor scalability and inefficient sampling. To address these limitations, we introduce the Entity-Aware Causal (EAC) framework, which tackles MARL communication from an entity-centric perspective. The core idea is to enhance communication efficiency through entity-aware communication target selection and causal inference belief mechanism, we make three main contributions. Firstly, we design an entity-aware hypernetwork that identifies communication targets based on individual state information and employs a masked-attention mechanism to enable scalable and sparse communication topology. Secondly, we propose a causal inference beliefs mechanism to strengthen the belief of the communication between entities and reduce redundant message exchanges. Finally, our algorithm outperforms baseline multi-agent cooperative reinforcement learning algorithms across SMAC, SMAC_v2, GRF, and MPE benchmarks. We further demonstrate the robustness of the algorithm across various network topologies and sparsity levels.