Model-Free Deep Reinforcement Learning with Multiple Line-of-Sight Guidance Laws for Autonomous Underwater Vehicles Full-Attitude and Velocity Control

Abstract

Autonomous underwater vehicles (AUVs) are increasingly utilized, driving the need for enhanced autonomy. Conventional proportional–integral–derivative (PID) algorithms require frequent control parameter adjustments under varying voyage conditions, which increases operational and experimental costs. To address this issue, a multiple line-of-sight guidance law integrated with a deep reinforcement learning control framework is proposed. This framework enables seamless switching among guidance modes, such as waypoint following, path following, and trajectory tracking, to achieve optimal attitude control. For comprehensive control of roll, pitch, yaw, and longitudinal velocity, an augmented-twin delayed deep deterministic policy gradient (A-TD3) algorithm streamlines the training of the control agent. It enables adaptation to large-range attitude variations using small-scale training data, thereby reducing computational costs for diverse missions. Simulations demonstrate the efficacy of the proposed approach: A-TD3 improves training speed by 30.8% while mitigating issues such as excessive rudder motion, poor operability, and high energy consumption across different missions. The attitude control experiments on the X-AUV prototype validate that A-TD3's control performance with PID method.