Optimal consensus model-free control for multi-agent systems subject to input delays and switching topologies

In this paper, the optimal consensus control problem of the discrete-time multi-agent systems with switching topologies and input delays is investigated by adopting the adaptive dynamic programming method. Through introducing a new state variable, the original input-delayed system can be transformed into a delay-free one. Then, a novel local performance index function is designed for each agent to eliminate the impact of switching topologies, which does not explicitly rely on the information of neighbors. Based on Bellman optimality principle, Lyapunov stability theorem and deep reinforcement learning method, the stability of the error system and the optimality of the value function are proved. In order to solve the consensus problem of the unknown systems, we propose a new value iteration algorithm based on the input and output data of the system, which can not only guarantee the achievement of consensus but also minimize the performance index function. Finally, two numerical simulations based on actor-critic neural networks are given, including the following two cases: periodic switching topologies and Markov switching topologies, to verify the effectiveness of the proposed optimal control scheme.