An optimal control method for uncertain process industry based on working condition relevance and policy transfer

Abstract

Energy consumption optimal control is a key challenge in process industries and is essential for enhancing production efficiency and reducing costs. However, the intricate physical and chemical reaction mechanisms in process industries make it difficult to establish precise process models. Moreover, unpredictable fluctuations in raw materials introduce uncertain variations in working conditions, causing unknown working conditions to emerge. Under this circumstance, the mismatch between the trained controller and the unknown working condition leads to deviation in control results. To get around these issues, a control method based on transfer reinforcement learning is proposed, which comprises two constituents. First, multiple RL controllers are trained under different working conditions to learn their distinct characteristics independently. By embedding domain knowledge into the reward function, the learning of control policies is guided to optimize energy consumption. Second, seeing that retraining RL controllers when encountering unknown working conditions is a costly endeavor, the concept of working condition relevance is proposed to quantify the degree of correlation between known and unknown working conditions. By utilizing a small amount of interaction experience with unknown and known working conditions, the control policies for known working conditions are transferred to the ones for unknown working conditions. As a result, the learning costs of the control policy are reduced while the negative transfer is mitigated to a certain extent. The simulation experiments on the zinc electrowinning process demonstrate that our method transfers control policies effectively when unknown working conditions arise, reducing learning costs noticeably.