Data-driven optimal bipartite containment control for heterogeneous multi-agent systems over coopetition networks

This paper investigates the optimal bipartite containment consensus control problem for heterogeneous multi-agent systems (HMASs) over coopetition networks. The primary challenge lies in achieving efficient bipartite containment control without requiring detailed knowledge of individual agent dynamics. To address this, we propose a distributed solution that employs model-free reinforcement learning. Our methodology considers both cooperative and competitive communication interactions, where followers are randomly allocated to two interconnected subnetworks that engage in both competition and collaboration with leaders. Specifically, in cooperative scenarios, followers converge toward the leaders’ convex hull, while in competitive cases, they converge to its antipodal counterpart. To approximate optimal control strategies, actor-critic neural networks (NNs) are utilized in conjunction with a policy iteration (PI) algorithm. The efficacy of the proposed method is validated through two simulation experiments.