Adversarial environment design for crowd navigation based on deep reinforcement learning

Abstract

The widespread use of mobile robots has increased the shared space between humans and robots, necessitating advanced solutions for crowd navigation. Recent studies have proposed approaches based on deep reinforcement learning to safely and efficiently achieve this goal. However, these approaches face challenges such as difficulty in presenting diverse pedestrian patterns and limited generalization performance. This study proposes a framework called Simultaneous training Process with Adversarial Crowd Environment (SPACE), which is an implemented artificial intelligence that generates crowd navigation environments. This framework competitively trains a crowd navigation agent and an adversarial crowd environment. In the adversarial crowd environment, the adversarial agent places pedestrians to induce collisions with the crowd navigation agent. By applying artificial intelligence within the episode-generation, this framework addresses vulnerabilities of previous approaches and allows the training of robust crowd navigation agents with high generalization performance. Experimental results demonstrate up to a 24.62% increase in navigation success rate and a 41.6% improvement in minimum distance from pedestrians compared to agents trained in non-adversarial environments, ensuring safer crowd navigation. Furthermore, SPACE exhibits more stable navigation performance in evaluation environment settings that are significantly more complex than the training scenarios. These findings highlight the promise of SPACE for training crowd navigation agents capable of operating effectively under diverse environmental conditions.