Safe reinforcement learning for optimization of batch processes with uncertainties: A Bayesian predictive exploration approach

Abstract

Optimization of batch processes is a challenging task due to their complex non-linear dynamics and various uncertainties. Recently, Reinforcement learning (RL) has been recognized as a promising alternative to solving this challenging problem. In this paper, we present a new safe RL method which is referred to as the Bayesian Predictive Exploration Approach. Firstly, the Bayesian neural networks (BNN) are introduced with variational mixture posteriors to represent the value function distributions, such that uncertainties can be more efficiently characterized. For the sake of safe explorations, we evaluate the profits and safety-risks by exploring multiple future decisions. The decisions are optimized to maximize the expected profit while avoiding constraint violations in the face of stochastic uncertainties. Both the expectations and variances of rewards and safety-risks are taken into considerations within the learning process. Finally, the effectiveness of the proposed approach is illustrated on two batch process examples.