LLM-guided contrastive role representation for diverse multi-agent coordination

Abstract

Multi-agent reinforcement learning (MARL) has achieved notable progress in cooperative tasks, yet the common use of parameter sharing often leads to behavioral homogeneity, which restricts the diversity and adaptability of agent coordination. To address this issue, we propose Contrastive Learning for Role Representation (CLRE), a framework designed to promote strategic diversity by encouraging agents to learn distinct and semantically meaningful roles. The key idea is to employ a Large Language Model (LLM) as an offline domain expert that automatically generates semantic role labels from agent states. These labels provide reliable supervision for a contrastive learning objective, guiding the model to acquire role embeddings that are both discriminative and interpretable. To ensure these role representations effectively support coordination, CLRE incorporates a graph attention network that captures inter-agent relationships and a dual-attention value function that integrates role information with the global state for context-aware decision-making. Experiments on the StarCraft Multi-Agent Challenge (SMAC) benchmark demonstrate that CLRE achieves superior win rates and sample efficiency compared with state-of-the-art MARL methods. Ablation studies further confirm that each component contributes to the overall performance, showing that the proposed approach enables diverse and efficient cooperation among agents in complex environments.