Optimal Output Feedback Learning Control for Continuous-Time Linear Quadratic Regulation

The classical linear quadratic regulation (LQR) problem of linear systems by state feedback has been widely addressed. However, the LQR problem by dynamic output feedback with optimal transient performance remains open. The main reason is that the observer error inevitably leads to suboptimal transient performance of the closed-loop system. In this article, we propose an optimal dynamic output feedback learning control approach to solve the LQR problem of linear continuous-time systems with unknown dynamics. In particular, we propose a novel internal dynamics called the internal model. Unlike the classical $p$-copy internal model, it is driven by the input and output of the system, and the role of the proposed internal model is to compensate for the transient error of the observer such that the output feedback LQR problem is solved with guaranteed optimality. A model-free learning algorithm is developed to estimate the optimal control gain of the dynamic output feedback controller. The algorithm does not require any prior knowledge of the system matrices or the system's initial state, thus leading to an optimal solution to the model-free LQR problem. The effectiveness of the proposed method is illustrated using an aircraft control system.