Real-world implementation of offline reinforcement learning for process control in industrial dividing wall column

Abstract

Reinforcement Learning (RL) has emerged as a promising approach for automating industrial process control, particularly in handling stochastic disturbances and complex dynamics. However, conventional RL methods pose significant safety concerns in real-world applications due to their reliance on extensive real-time interactions with the environment. Offline RL, which derives an optimal policy solely from historical operational data, provides a safer alternative but remains underexplored in industrial chemical processes. In this study, we apply Calibrated Q-Learning (Cal-QL), an offline-to-online RL algorithm, to temperature control of an industrial dividing wall column (DWC). We propose a practical procedure for deploying offline RL in chemical plants, integrating a Long Short-Term Memory (LSTM) network with a Deep Q-Network (DQN) to effectively process time series data structure and discrete action distributions commonly encountered in plant operations. Extensive simulation studies and real-world experiments on an industrial DWC demonstrate the suitability of the proposed framework. We also highlight the critical role of reward function design in balancing short- and long-term objectives, significantly influencing control performance. Our best performing configuration achieved stable temperature control with a high automation ratio of 93.11%, underscoring the feasibility and practical effectiveness of offline RL for complex industrial plant operations.