Model-free H∞ control of Itô stochastic system via off-policy reinforcement learning

Abstract

The stochastic $H_{\infty }$ control is studied for a linear stochastic Itô system with an unknown system model. It is known that the linear stochastic $H_{\infty }$ control issue can be transformed into the problem of solving a so-called generalized algebraic Riccati equation (GARE), which is a nonlinear equation that is typically difficult to solve analytically. Worse, model-based techniques cannot be utilized to approximately solve a GARE when an accurate system model is unavailable or prohibitively expensive to construct in reality. To address these issues, an off-policy reinforcement learning (RL) approach is presented to learn the solution of a GARE from real system data rather than a system model; its convergence is demonstrated, and the robustness of RL to errors in the learning process is investigated. In the off-policy RL approach, the system data may be created with behavior policies rather than the target policies, which is highly significant and promising for use in actual systems. Finally, the proposed off-policy RL approach is validated on a two-mass spring system.