Digital twin-driven reinforcement learning-based operational management for customized manufacturing

Abstract

Due to the increasing complexity of customer demands for different batches and types of products, manufacturing operations management has been facing the challenge of uncertain product arrival times and resource processing times in customized manufacturing (CM). This paper proposes a dynamic scheduling method to solve the uncertainty in CM via the integration of the digital twin and fuzzy reinforcement learning methods. In this study, a digital twin-driven framework is first designed to describe the operation management system (OMS) hierarchies. Then a semi-Markov decision process (MDP) model with fuzzy definition is built by abstracting the stochastic scheduling process. To solve the semi-MDP model, an asynchronous multi-edge co-training method is presented to train a fuzzy deep neural network through closed-loop control of virtual commissioning, illustrating how the digital twin-driven OMS adapts to dynamic production requirements. Finally, the proposed method is verified by the performance of comparative experiment. Experimental results show that for randomly arriving products, the proposed method guarantees timely training and scheduling decisions and has the highest total system profit compared to other competing methods (Hybrid Multi-Agent System Negotiation and Ant Colony Optimization (HMA), Onto_MDP, and Deep Q Networks (DQN)). Also, the proposed method shows better scheduling performance in terms of average decision time, average training time and number of finished products when resources are abnormal.