Data-driven robust linear quadratic regulator: A minimax design using reinforcement learning

This paper presents a minimax design approach based on model-free reinforcement learning (RL) to solve the robust linear quadratic regulator (LQR) problem. The proposed method derives a controller that guarantees stability and tends to be optimal as the data amount increases, even in the presence of unknown system dynamics. Initially, the robust LQR problem is transformed into a zero-sum differential game to minimize the worst linear quadratic cost in the system ensemble compatible with data containing Gaussian noise. Then, the RL algorithm is delineated, accompanied by the necessary and sufficient data condition for the existence of a unique solution, which is equivalent to the model-based minimax design solution. The RL-based controller can stabilize the original system with a high probability. Simultaneously, the sub-optimality gap between the controller’s performance and the optimal performance illustrates the asymptotic optimality of the controller. In addition to these attributes, the proposed method exhibits a distinct advantage in terms of computational efficiency. Simulations validate the effectiveness of the proposed method in terms of robustness, optimality, and run time.