Digital twin driven dynamic scheduling of discrete manufacturing workshop with transportation resource constraint using multi-agent deep reinforcement learning

In discrete manufacturing workshop where disturbances occur frequently, the dynamic scheduling problem that considers transportation resource constraint is complex and challenging. Additionally, rescheduling without evaluating the impact of disturbances may adversely affect production stability of workshop. To address these issues, this paper proposes a dynamic scheduling framework based on digital twin and deep reinforcement learning. Specifically, the digital twin environment is constructed to provide a high-fidelity training environment for scheduling agents and serve as a simulation means for evaluating the impact of disturbances. Furthermore, a rescheduling trigger discriminator mechanism is designed to dynamically determine the necessity of rescheduling. In particular, the multi-agent proximal policy optimization with multiple critics (MAPPO-MC) is proposed to efficiently solve the discrete manufacturing workshop dynamic scheduling problem with transportation resource constraint. The innovation of MAPPO-MC lies in using the global critic to facilitate collaboration among scheduling agents from a global perspective, while employing individual critics to guide corresponding agents to learn their specialized scheduling knowledge from a local perspective, thereby achieving optimal scheduling decisions. Finally, extensive experiments have demonstrated the effectiveness of the proposed framework and the superiority of the MAPPO-MC. Under this framework, MAPPO-MC can respond promptly and effectively to disturbances in the workshop while ensuring stable production.