Graph neural network and expert-guided deep reinforcement learning for solving flexible job-shop scheduling problem

Abstract

Flexible Job-shop Scheduling Problem (FJSP) is crucial for efficient and flexible automated production. Recent advancements in Reinforcement Learning (RL) have shown promise in solving sequential decision problems. However, most Deep Reinforcement Learning (DRL) algorithms rely on Priority Dispatching Rules (PDRs), which limits scheduling efficiency. This paper proposed a novel Graph Neural Network and Expert-Guided Deep Reinforcement Learning (GNN-EGDRL) framework that employs Graph Neural Network (GNN) to extract features from both machines and operations, thereby integrating operation selection and machine assignment into a unified composite decision-making process. Expert solutions generated by PDRs guide the initial stages of training, allowing agents to gradually transition to self-directed action selection, thus balancing exploration and exploitation. The expert guidance strategy emphasizes the relationships between operations and machines, enhancing the model’s sensitivity and providing granular guidance for feature extraction, leading to optimized solutions. Extensive experiments demonstrate that the proposed GNN-EGDRL method consistently outperforms traditional PDRs and other end-to-end DRL over all problem instances. Notably, this superiority has been validated in a wide range of scenarios, including larger-scale examples, underscoring the method’s scalability and robustness. Furthermore, the method exhibits strong performance in scenarios not encountered during training, highlighting its effectiveness and adaptability in diverse production environments.