Online adaptive model‐based stochastic control with limited data: A case study with a colloidal self‐assembly system

Abstract

Data‐driven optimal control is receiving increasing attention in colloidal self‐assembly. While model‐based control can suffer from predictive model errors, model‐free control such as reinforcement learning can struggle with policy training which can result in waste of materials. Here we proposed an online adaptive model‐based stochastic model predictive control framework to address the challenge of limited data for predictive model development. We evaluated its performance on a colloidal self‐assembly system using Markov state models as the surrogate physical system. Results showed that online adaptation can substantially improve the prediction accuracy of an initially inaccurate model, enabling stochastic model predictive control to yield satisfactory performance. We further found that the control performance is dependent on objective function design and improved overall model accuracy does not necessarily guarantee better control. We anticipate the framework to also be applicable to other systems, where obtaining large amount of data for model development is challenging.