Learning Based Optimal Sensor Selection for Linear Quadratic Control with Unknown Sensor Noise Covariance *

Abstract

In this article, an optimal sensor selection problem is considered under the framework of linear quadratic control. The objective is to find the best strategy of selecting one sensor among a set of sensors at each time step so that the expected system performance is minimized over multiple time steps. This problem is formulated as a multi-armed bandit problem. Uncertainties are captured through noisy sensor measurements, which account for the performance deterioration caused by unknown sensor noise covariance. In this context, several action-value based reinforcement learning methods are proposed to evaluate the performance of different sensor selection strategies. Moreover, a statistical method is developed to estimate the unknown sensor noise covariance as a byproduct. The almost sure convergence to the true sensor noise covariance is guaranteed as the number of times a sensor being selected goes to infinity. A linear quadratic control example is presented to illustrate the proposed approaches and to demonstrate their effectiveness.