Graph-QMIX: Addressing the Partial Observation Issues via Graph Neural Network in Multi-Agent Reinforcement Learning

Abstract

In recent years, with the development of multiagent reinforcement learning, more and more complex tasks have been solved. However, today’s multi-agent reinforcement learning faces two challenges: 1) the global state is always used to train the neural network, which is hard to obtain in the real-world; 2) compared to the global state, concatenating local observations decreases the performance of multi-agent reinforcement learning algorithms. These challenges make it difficult to apply multi-agent reinforcement learning algorithms in real-world scenarios. To solve these challenges, we proposed the Graph-QMIX algorithm, where all agents are seen as a graph, and the graph convolutional neural network is used to integrate the local observations of the agents. We evaluate our method in map 2s vs lsc and map 10m vs 11m of SMAC environment. Empirically simulation results show that our method reaches a strong performance as much as QMIX using the global state, and is much stronger than QMIX using the concatenating local observations.