Job shop scheduling by Deep Dual-Q Network with Prioritized Experience Replay for resilient production control in flexible manufacturing system

Abstract

The increasing demand for mass customization and intensifying market competition have made the production cycles shorter and the product iterations faster. As a result, most products are in small and medium batches, introducing both opportunities and challenges to conventional job scheduling. The resilient production control in flexible manufacturing system focuses on creating adaptive and sustainable systems that present the characteristics of multi-product-type variant-volume discrete-flow mixed-flow production, which is still challenging to balance flexibility and efficiency in production. In this paper, a Deep Dual-Q Network with Prioritized Experience Replay (DDQN-PER) approach is proposed to solve the job shop scheduling problem (JSSP). It combines the advantages of the Dueling and Double DQN architecture, utilizing prioritized replay and neural networks to approximate state–action (Q). To extract and store experience data from the experience memory more efficiently, the states of shop environment are represented as information matrices. The two-phase algorithm, comprising iterative offline training and online application (OTOA), trains scheduling policies, forming a dynamic closed loop between offline scheduling results and online real-time production control. Case study and downtime experiments conducted on key machines validate the superiority of the proposed approach. Experimental results demonstrate that using DDQN-PER with optimized hyper-parameters effectively solves the JSSP.