Stochastic H2/H∞ off-policy reinforcement learning tracking control for linear discrete-time systems with multiplicative noises

Abstract

This paper focuses on an infinite horizon mixed $H_2/H_{\infty }$ model-free optimal tracking control problem for linear discrete-time stochastic systems with multiplicative noises. A model-based algorithm for mixed $H_2/H_{\infty }$ control of the systems is first proposed to derive the optimal tracking control gain. Then, an off-policy reinforcement learning (RL) algorithm is proposed to solve the stochastic algebraic Riccati equation (SARE) online by collecting a set of sample-path measurement data along the system trajectory. It is further proven that the off-policy RL algorithm is equivalent to the model-based algorithm, and the off-policy RL algorithm not only dose not need to specify external disturbances but also ensures that the probing noise does not bias the algorithm’s convergence. The effectiveness of the proposed control scheme is verified by using a cart-inverted pendulum system.