A Digital Twin-Based Production-Maintenance Joint Scheduling Framework with Reinforcement Learning

Abstract

The bridge of job scheduling and production equipment maintenance is usually the main joint scheduling task of a production system. However, the predicament of data acquisition in real systems leads to the difficulty of verifying the effectiveness of scheduling algorithms. In order to make joint scheduling work easier to implement in real production systems, this paper presents a joint scheduling framework for production systems based on digital twin and reinforcement learning. Firstly, the virtual mapping of physical production system, namely digital twin system, is established by using AnyLogic software and multi-agent modeling technology. Then, a joint scheduling agent is trained by Deep Q Network (DQN) algorithm and the virtual data generated by the twinning system. And the experimental results demonstrate the effectiveness of proposed framework in production systems with uncertainties, and it has higher production efficiency and lower machine failure frequency compared with a scheduling scheme based on common-used heuristic rules.