IDSI-SO: A Multiagent Safe Reinforcement Learning Navigation Based on Threat Assessment in Complex Dynamic Environments

Abstract

Achieving efficient and safe navigation of multiple agents in complex environments is significant for future transportation. Challenges of multiagent navigation are as follows: first, it is difficult to obtain safe navigation policies when the dimension of observation/action space increases rapidly with the number of agents; second, the algorithm needs to deal with complex static obstacles in realistic scenes; third, the interaction avoidance between agents is not clear. In this work, we propose the IDSI-SRL-ORCA (IDSI-SO for short hereafter), a multiagent safe reinforcement learning (SRL) navigation method. Interactive driving safety index (IDSI) is a general method for analyzing observation in navigation and reducing its spatial dimensions. It performs a threat assessment based on observations of dynamic obstacles. IDSI considers the effect of interaction between agents on safety estimation and reduces the computation of field intensities in DSI [1]. IDSI-SO uses threat attention and opponent modeling to extract useful information from raw sensors based on threat assessment from IDSI. This reduces the dimensionality of observations for reinforcement learning and ultimately improves training efficiency. Experiments show that IDSI-SO reduces the average threat by 13.4% compared to SRL-ORCA [2] and obtains a success rate of 94.5% in complex scenarios. Video is available at https://www.youtube.com/watch?v=MS7MvA3gpS4.