An Adaptive Job Shop Scheduling Mechanism for Disturbances by Running Reinforcement Learning in Digital Twin Environment

Abstract

Practical manufacturing system operates in highly dynamic and uncertain environments, where stochastic disturbances disrupt the execution of production schedule as originally developed. Previous dynamic scheduling mainly focuses on the constructing predictive models for machine unavailability, with little studies on the adaptive and self-learning capacities for changing scheduling environments. Therefore, a digital twin (DT) driven scheduling with dynamic feedback mechanism is proposed, in which a reinforcement learning (RL) based adaptive scheduling is developed in DT to make corrective decisions for the disturbances during production runs. In the proposed architecture, the happening disturbance is firstly detected in the virtual layer by the status continuously updating in accordance with the physical workshop. Furthermore, the reschedule triggering condition is determined in real-time through quantization of the progress deviations resulting from disturbances. For the scheduling approach, the multi-agents RL (MARL) based adaptive scheduling method is built to perceive the dynamic production status from virtual environment and implement corrective strategies to hedge against the occurred disturbances. Finally, the proposed method is verified by a practical job shop case and the corresponding DT system is developed to show the effectiveness and advantages after a practical implementation.