On Model-free Reinforcement Learning for Switched Linear Systems: A Subspace Clustering Approach

Abstract

In this paper, we study optimal control of switched linear systems using reinforcement learning. Instead of directly applying existing model-free reinforcement learning algorithms, we propose a Q-learning-based algorithm designed specifically for discrete time switched linear systems. Inspired by the analytical results from optimal control literature, the Q function in our algorithm is approximated by a point-wise minimum form of a finite number of quadratic functions. An associated update scheme based on subspace clustering for such an approximation is also developed which preserves the desired structure during the training process. Numerical examples for both low-dimensional and high-dimensional switched linear systems are provided to demonstrate the performance of our algorithm.