A data-ensemble-based approach for sample-efficient LQ control of linear time-varying systems

Abstract

This paper presents a sample-efficient, data-driven control framework for finite-horizon linear quadratic (LQ) control of linear time-varying (LTV) systems. In contrast to the time-invariant case, the time-varying LQ problem involves a differential Riccati equation (DRE) with time-dependent parameters and terminal boundary constraints, complicating data-driven control. Additionally, the time-varying dynamics invalidate the use of the Fundamental Lemma. To overcome these challenges, we formulate the LQ problem as a nonconvex optimization problem and conduct a rigorous analysis of its dual structure. By exploiting the inherent convexity of the dual problem and analyzing the KKT conditions, we derive an explicit relationship between the optimal dual solution and the parameters of the associated Q-function in time-varying case. This theoretical insight supports the development of a novel, sample-efficient, non-iterative semidefinite programming (SDP) algorithm that directly computes the optimal sequence of feedback gains from an ensemble of input-state data sequences without requiring model identification or a stabilizing controller. The resulting convex, data-dependent framework provides global optimality guarantees for completely unknown LTV systems. As a special case, the method also applies to finite-horizon LQ control of linear time-invariant (LTI) systems. In this setting, a single input-state trajectory suffices to identify the optimal LQ feedback policy, improving significantly over existing Q-learning approaches for finite horizon LTI systems that typically require data from multiple episodes. The approach provides a new optimization-based perspective on Q-learning in time-varying settings and contributes to the broader understanding of data-driven control in non-stationary environments. Simulation results show that, compared to recent methods, the proposed approach achieves superior optimality and sample efficiency on LTV systems, and indicates potential for stabilizing and optimal control of nonlinear systems.