Results on sparsity and estimation accuracy in Orthogonal Matching Pursuit with application to optimal input design

Abstract

Appropriate excitation conditions are essential for reliably identifying sparse models. In regression problems, these conditions are often characterized by properties of the regressor matrix, with mutual coherence, the maximum correlation between regressors, playing a central role in enabling sparse recovery. However, obtaining sparse estimates is rarely the sole objective, as estimation accuracy is also important. When a model is used in an application, e.g. control design, acceptable performance with high probability can be (approximately) ensured by requiring the confidence ellipsoid to be contained in a certain ellipsoid which depends on the performance specifications in the application. However, it is well known that experiments fulfilling such requirements using minimal excitation energy (least-costly experiments) tend to generate highly correlated regressors, in conflict with the requirements for sparsity. Adhering to this setting and focusing on the popular orthogonal matching pursuit algorithm (OMP), we derive conditions for simultaneously ensuring sparsity and having the confidence ellipsoid contained in a pre-specified ellipsoid. We extend this result to a recently proposed two stage sparse estimation method where a linear transformation is used in a pre-processing step before OMP to reduce mutual coherence. A final contribution is to show that our theoretical results are of importance in optimal input design for sparse models. Specifically, we show that the choice of hyperparameters in a recently proposed input design method can be guided by our contributions and we show explicitly how this can be done in this two-stage method.