Hierarchical reinforcement learning for dynamic collision avoidance of autonomous ships under uncertain scenarios

Abstract

Autonomous ships hold substantial potential for enhancing navigational safety, improving collision avoidance efficiency, and increasing adaptability in complex maritime environments, thereby presenting broad prospects for intelligent shipping. This paper introduces a dynamic collision avoidance control method based on a hierarchical reinforcement learning framework for autonomous ships. By integrating high-level global intent planning with low-level fine-grained rudder control, the proposed approach markedly enhances the interpretability, stability, and behavioral consistency of the learned policy. Furthermore, a multidimensional uncertainty modeling mechanism is incorporated during training, systematically accounting for variations in initial states and obstacle behavior patterns, which effectively strengthens policy adaptability and generalization under uncertain conditions. To validate the method, simulations are conducted in representative encounter scenarios as well as in omnidirectional dynamic obstacle tests. A comprehensive evaluation is carried out using multiple control performance metrics, environmental adaptability analysis, policy consistency assessment, and equivalent energy consumption comparisons. The results confirm that the proposed approach achieves stable and reliable intelligent collision avoidance control in highly dynamic environments, offering a feasible and scalable solution for high-performance collision avoidance in intelligent maritime navigation.