Adversarial dynamic games for Markov jump systems: A policy iteration Q-learning method

Abstract

This paper presents a reinforcement Q-learning approach for solving adversarial dynamic games in Markov jump systems. The $H_{\infty }$ control problem is first formulated as a two-player zero-sum dynamic game, where the control policy and the disturbance policy act as adversarial players. To derive the Nash equilibrium control strategies for such games, a set of coupled algebraic Riccati equations is established, with the disturbance attenuation level properly prescribed. On this basis, two novel data-driven parallel Q-learning algorithms are proposed. The advantages of the proposed method are threefold: (i) it does not require precise knowledge of the system dynamics; (ii) it learns the optimal disturbance attenuation level; (iii) it yields Nash equilibrium control strategies. Finally, two simulation examples validate the effectiveness of the proposed method.