Distributed Nash equilibrium solution for multi-agent game in adversarial environment: A reinforcement learning method

Abstract

This paper investigates the leader–follower optimal consensus problem for linear multi-agent systems with adversarial inputs from a differential graphical game perspective. For the multi-agent optimal control problem described by differential graphical game, it is equally significant that the control policy is distributed and adheres to Nash equilibrium solution. However, achieving both distributed control and Nash equilibrium simultaneously has proven impossible in most existing game formulations. This paper proposes a new game formulation that can overcome this limitation, enabling each agent to reach a Nash equilibrium under a distributed policy, thereby improving system performance. Furthermore, a partially model-free reinforcement learning method is employed to obtain the optimal policy when the dynamic information is partially unknown, with admissibility condition of the initial policy further relaxed. Finally, two comparative simulations are presented to demonstrate the validity and superiority of the proposed approach.