Learning Based Optimal Sensor Selection for Linear Quadratic Control with Unknown Sensor Noise Covariance *

In this article, an optimal sensor selection problem is considered under the framework of linear quadratic control. The objective is to find the best strategy of selecting one sensor among a set of sensors at each time step so that the expected system performance is minimized over multiple time steps. This problem is formulated as a multi-armed bandit problem. Uncertainties are captured through noisy sensor measurements, which account for the performance deterioration caused by unknown sensor noise covariance. In this context, several action-value based reinforcement learning methods are proposed to evaluate the performance of different sensor selection strategies. Moreover, a statistical method is developed to estimate the unknown sensor noise covariance as a byproduct. The almost sure convergence to the true sensor noise covariance is guaranteed as the number of times a sensor being selected goes to infinity. A linear quadratic control example is presented to illustrate the proposed approaches and to demonstrate their effectiveness.