Finite-time regret minimization for linear quadratic adaptive controllers: An experiment design approach

Abstract

We tackle the problem of finite-time regret minimization in linear quadratic adaptive control. Regret minimization is a scientific field in both adaptive control and reinforcement learning research communities which studies the so-called trade-off between exploration and exploitation. Even though a large focus has been on linear quadratic adaptive control with theoretical finite-time bound guarantees on the expected regret growth rate, most of the proposed optimal exploration strategies do not take into account the scaling constant associated with the growth rate. Moreover, the exploration strategies are limited to white noise excitation. Using tools from experiment design, we propose a computationally tractable solution for the design of the external excitation chosen as a white noise filtered by a finite impulse response filter which is adapted on-line. In a numerical example it is shown that this approach results in a lower regret in comparison with available strategies.