Two-dimensional reinforcement learning model-free fault-tolerant control for batch processes against multi- faults

Abstract

Aiming at the characteristics of batch process changing along with time and batch directions, the existence of unmodeled dynamics, and the partial failure of actuators or/and sensors, we propose a novel 2D reinforcement learning (RL) fault tolerant control strategy without considering model parameters. Firstly, a two-Dimensional (2D) augmented state space model and 2D Q function-based fault tolerant control (FTC) framework is established. The 2D Bellman equation can be acquired by analyzing the relationship between the 2D value function and the 2D Q function. Based on the extended model and Q-learning concept of RL, a data-driven FTTC independent of model parameters is designed, and a 2D data-driven Q-learning algorithm is proposed. Finally, taking the pressure holding phase in the injection process as the experimental object, the control effect is compared with that of the traditional model-based FTC, and better tracking performance and unbiasedness to the probing noise can be proved.