Reinforcement learning based iterative learning control for nonlinear batch process with non-repetitive uncertainty via Koopman operator

Abstract

To tackle the time and batchwise uncertainty often involved in nonlinear batch process, this paper proposes a deep reinforcement learning (DRL) based ILC scheme via Koopman operator. Using the Koopman operator, the original nonlinear system is reformulated into a high-dimensional linear space form. Then, a DRL agent with neural network is introduced into the 2D ILC framework to compensate for non-repetitive uncertainty. Correspondingly, a synthetic 2D ILC-DRL scheme is designed to improve the system tracking performance against time and batchwise uncertainty. Meanwhile, the convergence conditions of the proposed ILC scheme are analyzed with a proof through the linear matrix inequality. An illustrative example of continuous stirring tank reactor (CSTR) demonstrates that the established high-dimensional linear model can ensure good accuracy compared to the original nonlinear process model, with an output error of smaller than 5%. Moreover, the tracking error is significantly reduced over 90% by the reinforcement learning based ILC in comparison with the recently developed dynamic iterative linearization and PD-type ILC methods.