SUNRISE: multi-agent reinforcement learning via neighbors’ observations under fully noisy environments

Multi-agent reinforcement learning (MARL) methodologies have achieved notable advancements across diverse domains. Despite these successes, the susceptibility of neural networks to perturbed data and the ubiquity of external attacks in real-world settings, such as sensor noise, pose challenges for MARL approaches. The pivotal issue revolves around the effective transfer of policies learned in idealized simulation environments to the complexities inherent in real-world scenarios. More precisely, when agents are unable to obtain any accurate observations of the external environment throughout the entire policy learning process, the MARL methods cannot learn effective policies. In addressing this issue, we propose a methodology wherein noisy observations from neighboring agents are utilized, with an agent’s own noisy observations serving as surrogate ground truth. This approach facilitates the learning of effective policies by MARL methods in environments characterized by pervasive noise. We design a denoising representation network to filter out the principal state information from environment data characterized by noise to mitigate the adverse effects of noise on the process of policy learning. Then, we integrate the denoising representation network with classic MARL methodologies to learn effective policies within environments characterized by pervasive noise. A series of exhaustive experimental results demonstrate the efficacy of our approach in attenuating the impact of external attacks on the optimization parameters of neural networks during the policy-learning process. Moreover, our methodology exhibits compatibility with classic MARL methods, allowing for the learning of effective policies.