A novel deep reinforcement learning framework based on digital twins for dynamic job shop scheduling problems

With the increasing diversity of product demands, the complexity of production scheduling planning has also been continuously escalating. Existing scheduling models exhibit significant deviations from actual production systems, making it challenging to directly apply scheduling algorithms to practical systems. High-fidelity digital twin (DT) models offer the capability to faithfully replicate production processes, providing effective means for training and validating scheduling algorithms. In this context, we propose a dynamic scheduling framework, DT-DRL, based on DT for deep reinforcement learning (DRL) applications in real production scheduling. Firstly, we employ DT technology to model actual production lines, effectively addressing the issue of model completeness. Secondly, we utilize the Double Deep Q-Network (DDQN) algorithm for offline training, followed by online decision-making, effectively addressing the challenge of real-time dynamic scheduling. Lastly, experimental training and validation are conducted using historical order and equipment data from the water heater inner tank welding production line. The experimental results demonstrate the robustness of our model.