Reinforcement-learning-based control framework for leader-following cascade formation of multiple perturbed surface vehicles

Abstract

This article presents a novel reinforcement learning (RL)-based leader/follower cascade formation control framework for a fleet of perturbed surface vehicles (SVs), aiming to achieve accurate trajectory tracking while minimizing a predefined cost function to enhance system performance. The proposed distributed control structure explicitly accounts for both the dynamic and kinematic subsystems of each SV agent. It features a hierarchical control design, where the outer loop generates high-level desired trajectories for each vehicle, and the inner loop employs an actor/critic RL strategy to derive the optimal low-level control scheme. In contrast to conventional back-stepping-based formation control methods, this approach independently addresses the dynamic and kinematic subsystems of each SV agent, enabling a more modular and robust analysis. Rigorous stability proofs are provided to ensure that tracking errors and learning errors for both individual agents and the overall closed-loop control system are uniformly and ultimately bounded (UUB). To validate the effectiveness and practicality of the proposed strategy, simulation studies are conducted with a group of five SVs, demonstrating superior performance and feasibility in achieving robust formation control under perturbations.