Hierarchical RL-ESN control for autonomous underwater vehicle: 6-DOF large angle rotation maneuvering

Abstract

The attitude representation of the autonomous underwater vehicle (AUV) typically relies on Euler angles and unit quaternions. However, existing works have demonstrated that both approaches exhibit inherent limitations when solving large-angle rotation maneuvers, primarily characterized by singularities and unwinding phenomena. In light of these limitations, this article investigates a model-free tracking control scheme for AUV to perform arbitrary large-angle rotation maneuvers in six degrees of freedom (6-DOF). Specifically, a rotation matrix-based error dynamics is constructed to achieve a globally unique attitude representation. A key technical obstacle is that defining attitude error directly via the rotation matrix complicates controller design. To circumvent this problem, an alternative error metric is presented to transform the rotation matrix-based attitude error from the special orthogonal group in three-dimensional space (SO(3)) to Euclidean space. Distinguished from a single estimation network, a hierarchical echo state network (ESN) is established to estimate hydrodynamic coefficients and actuator faults, which comprises multiple independent subnetworks to conduct various estimation tasks. Moreover, a collaborative critic network (CCN) is utilized to generate reinforcement signals, endowing the hierarchical ESN with favorable learning capability. In this way, the system robustness is significantly enhanced while ensuring high tracking performance. Finally, rigorous theoretical analysis and numerical simulations are presented to show the effectiveness and superiority of the proposed control scheme.