A multi-agent deep reinforcement learning method for fully noisy observations

Abstract

Multi-agent reinforcement learning (MARL) algorithms have achieved great breakthroughs in many aspects. The MARL algorithms can learn effective policies in ideal simulation environments. But different from the ideal simulation environments, noise is unavoidable in the real world. MARL algorithms need to learn effective policies in unavoidable fully noisy environments. In this paper, we consider a challenging multi-agent reinforcement learning problem: All agents cannot observe any noiseless observations from environments during the whole training process and MARL algorithms cannot learn effective policies in these fully noisy observation environments. To solve this problem, we propose a method called Robust $P$olicy $L$earning under Fully Noisy Observation vi$A$ De$N$oising R$E$presentation Ne$T$work (PLANET), which enables MARL algorithms learning effective policies in fully noisy observation environments. The PLANET method learns the effective policy through two steps. (1) Extracting the noise characteristics and motion laws to obtain clean observations information from fully noisy observation histories. (2) Making MARL algorithms extract information from the noise characteristics and motion laws information, and learn effective policies. The results of a series of exhaustive experiments show that our method can mitigate the effects of noise and learn effective policies in fully noisy observation environments. Our Artificial Intelligence contribution lies in introducing the denoising representation network that learns noise characteristics and motion dynamics to recover clean observations from fully noisy observations. The proposed PLANET framework could be applied to real-world multi-agent robotic and sensor network systems, potentially improving policy robustness under fully noisy observation.