End-to-End Multitarget Flexible Job Shop Scheduling With Deep Reinforcement Learning

Abstract

Modeling and solving the flexible job shop scheduling problem (FJSP) is critical for modern manufacturing. However, existing works primarily focus on the time-related makespan target, often neglecting other practical factors, such as transportation. To address this, we formulate a more comprehensive multitarget FJSP that integrates makespan with varied transportation times and the total energy consumption of processing and transportation. The combination of these multiple real-world production targets renders the scheduling problem highly complex and challenging to solve. To overcome this challenge, this article proposes an end-to-end multiagent proximal policy optimization (PPO) approach. First, we represent the scheduling problem as a disjunctive graph (DG) with designed features of subtasks and constructed machine nodes, additionally integrating information of arcs denoted as transportation and standby time, respectively. Next, we use a graph neural network (GNN) to encode features into node embeddings, representing the states at each decision step. Finally, based on the vectorized value function and local critic networks, the PPO algorithm and DG simulation environment iteratively interact to train the policy network. Our extensive experimental results validate the performance of the proposed approach, demonstrating its superiority over the state-of-the-art in terms of high-quality solutions, online computation time, stability, and generalization.