Competitive Control via Online Optimization with Memory, Delayed Feedback, and Inexact Predictions

Abstract

Recently a line of work has shown the applicability of tools from online optimization for control, leading to online control algorithms with learning-theoretic guarantees, such as sublinear regret. However, the predominant benchmark, static regret, only compares to the best static linear controller in hindsight, which could be arbitrarily sub-optimal compared to the true offline optimal policy in non-stationary environments. Moreover, the common robustness considerations in control theory literature, such as feedback delays and inexact predictions, only have little progress in the context of online learning/optimization guarantees. In this talk, based on our three recent papers, I will present key principles and practical algorithms towards online control with competitive ratio guarantees, which directly bound the suboptimality compared to the true offline optimal policy. First, I will show the deep connections between a novel class of online optimization with memory and online control, which directly translates online optimization guarantees to online control guarantees and gives the first constant-competitive policy with adversarial disturbances [1]. Second, I will analyze the performance of the most popular online policy in the control community, Model Predictive Control (MPC), from the online learning's perspective, and show a few important fundamental limits. Our results give the first finite-time performance guarantees for MPC [3]. Finally, I will discuss the influence of delayed feedback and inexact predictions on competitive ratio analysis [2].