Off-policy Q-learning-based Tracking Control for Stochastic Linear Discrete-Time Systems

In this paper, an adaptive optimal control is investigated for a stochastic linear discrete-time system with multiplicative state-dependent noise and control-dependent noise without knowledge of the system dynamics. With the framework of Q-learning, an off-policy state feedback solution for stochastic linear quadratic tracking (SLQT) problem has been proposed. First, an augmented system of the original system and the reference command generator is established to solve SLQT problem. Then, we present an optimal control by solving stochastic algebraic Riccati equation (SARE). Next, we present the on-policy and off-policy algorithms to achieve an adaptive optimal control without knowing the system dynamics. Finally, a simulation test is finally setup to verify the performance of the proposed adaptive optimal control.