Memory-based deep reinforcement learning for COLREGs-compliant obstacle avoidance in USV with limited environmental knowledge

Abstract

The deployment of collision avoidance algorithms for unmanned surface vehicles is often limited by their capabilities to perceive dynamic and unpredictable marine environments. Influenced by human memory mechanisms, a memory-based deep reinforcement learning (MDRL) algorithm is proposed in this study for autonomous collision avoidance given limited environmental knowledge. A memory space is established to archive historical navigation data, and gated recurrent units are used to integrate these data into short-term memory for network decision-making. Consequently, the algorithm substantially facilitates the optimization of short-term memory and immediate decision-making, further compensating for the deficiencies of immediate perceptual data, enabling precise evaluation of encounter scenarios and the development of avoidance strategies compliant with the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs). Experimental results demonstrate that the MDRL algorithm significantly enhances the collision avoidance capabilities of USV with limited environmental knowledge while ensuring COLREGs compliance.