Distributed Pursuit-Evasion Game of Limited Perception USV Swarm Based on Multiagent Proximal Policy Optimization

Abstract

This article proposes a distributed capture strategy optimization method for the pursuit-evasion game involving multiple unmanned surface vehicles. Considering the limited perception range of each pursuer, a multiagent proximal policy optimization method combined with a novel velocity control mechanism is utilized to guide the pursuers in approaching the evader and form a dynamic encirclement. Moreover, to facilitate deep reinforcement learning (DRL) training, a bidirectional gated recurrent unit feature network is constructed to extract the fixed-length vector representations from the variable-length observation sequences. In terms of the policy training, by employing virtual barriers and curriculum learning techniques during the training process, the generalization capabilities and convergence speed of the policy have been further improved. Finally, our method is compared with the other DRL methods through the comparative simulation experiments and virtual reality scene testing based on the gazebo three dimensional physics engine, verifying its significant advantages in the policy convergence speed, capture efficiency, and generalization capabilities.