Output Regulation Based on Zero-Sum Game for Discrete-Time System Driven by Exogenous Signal

This article proposes a novel Q-learning algorithm that relies solely on input-output data to address the output regulation control problem of complex discrete-time systems affected by exogenous signals. Unlike traditional methods, this algorithm does not require detailed system information, state knowledge, or data about external systems or exogenous signals. Additionally, the control strategy does not depend on state information, but on input-output data processed by a set of filters. We provide upper and lower bounds on the discount factor, eliminating the need to solve the Riccati equation. These bounds ensure that the value function remains finite, and we prove the stability of the system when using control inputs derived from the value function with the given discount factor. Furthermore, the Q-learning algorithm, when applied with input data containing probing noise, is shown to yield Q-function estimates that are independent of the probing noise. Finally, a simulation involving a grid-connected inverter is presented, demonstrating the effectiveness of the proposed algorithm in a practical setting.