Finite Sample Identification of Low-Order LTI Systems via Nuclear Norm Regularization

Abstract

This paper studies the problem of identifying low-order linear time-invariant systems via Hankel nuclear norm (HNN) regularization. This regularization encourages the Hankel matrix to be low-rank, which corresponds to the dynamical system being of low order. We provide novel statistical analysis for this regularization, and contrast it with the unregularized ordinary least-squares (OLS) estimator. Our analysis leads to new finite-sample error bounds on estimating the impulse response and the Hankel matrix associated with the linear system using HNN regularization. We design a suitable input excitation, and show that we can recover the system using a number of observations that scales optimally with the true system order and achieves strong statistical estimation rates. Complementing these, we also demonstrate that the input design indeed matters by proving that intuitive choices, such as i.i.d. Gaussian input, lead to sub-optimal sample complexity. To better understand the benefits of regularization, we also revisit the OLS estimator. Besides refining existing bounds, we experimentally identify when HNN regularization improves over OLS: (1) For low-order systems with slow impulse-response decay, OLS method performs poorly in terms of sample complexity, (2) the Hankel matrix returned by regularization has a more clear singular value gap that makes determining the system order easier, (3) HNN regularization is less sensitive to hyperparameter choice. To choose the regularization parameter, we also outline a simple joint train-validation procedure.