A Homotopy Method for Continuous-Time Model-Free LQR Control Based on Policy Iteration

Abstract

In recent years, reinforcement learning control theory has been well developed. However, model-free value iteration needs many iterations to achieve the desired precision, and model-free policy iteration requires an initial stabilizing control policy. It is significant to propose a fast model-free algorithm to solve the continuous-time linear quadratic control problem without an initial stabilizing control policy. In this paper, we construct a homotopy path on which each point corresponds to an linear quadratic regulator problem. Based on policy iteration, model-based and model-free homotopy algorithms are proposed to solve the optimal control problem of continuous-time linear systems along the homotopy path. Our algorithms are speeded up using first-order differential information and do not require an initial stabilizing control policy. Finally, several practical examples are used to illustrate our results.