Deep reinforcement learning for collision avoidance in unmanned surface vehicles: State-of-the-art

In recent years, deep reinforcement learning (DRL) has attracted growing attention in the field of collision avoidance for unmanned surface vehicles (USVs). The collision avoidance problem can be naturally formulated as a Markov decision process (MDP), where sequential decision-making is required under dynamic and uncertain maritime conditions. By integrating the perception capabilities of deep learning with the decision-making strengths of reinforcement learning, DRL demonstrates strong adaptability and robustness in solving such MDP-based problems. This review systematically summarizes the frameworks and principles of DRL algorithms for USVs collision avoidance, with emphasis on value-based methods, policy-based methods, and multi-agent DRL methods. Key research topics are systematically reviewed, including reward function design, state-space representation, exploration strategies, integration of safety regulations, robustness to disturbances, environment modeling, and application domains. Challenges in current studies are analyzed, and potential future research directions are proposed to advance DRL methodologies and promote their practical deployment in USVs navigation. This work aims to provide researchers with a consolidated understanding of DRL-based collision avoidance, fostering innovation and accelerating the engineering implementation of autonomous maritime systems.