Multi-UAV Optimal Formation Control via Actor-Critic Reinforcement Learning Algorithm

In this paper, the multi-agent synchronous actor-critic algorithm is developed to solve the optimal formation control problem of the disturbed multi-unmanned aerial vehicle system. Based on the optimal control theory, the optimal formation problem is transformed to seek the optimal solutions of a set of coupled Hamilton-Jacobi-Bellman equations. The multi-agent reinforcement learning algorithm via actor/critic structure is adapted to approximate such solutions. The adaptive tuning laws are given for both critic and actor networks, which ensure the approximate convergence of the optimal value and optimal controller and the stability of the closed-loop formation error system. The simulation is provided to verify the effectiveness of the proposed theoretical results.