A Tree neural network deep reinforcement learning for flexible job shop scheduling with transportation constraints

Abstract

The Flexible Job Shop Scheduling Problem with Transportation Constraints (FJSP-T) is critical for improving productivity in flexible manufacturing systems, particularly when automated guided vehicles (AGVs) are involved. This study focuses on minimizing the makespan by formulating the FJSP-T as a mixed-integer linear programming (MILP) model with explicitly defined constraints and objective functions. To solve large-scale instances efficiently, the problem is further modeled as a Markov Decision Process (MDP), where an agent sequentially selects operations, assigns machines, and allocates AGVs based on the current production state. A key contribution of this work is the development of a novel tree-based deep reinforcement learning algorithm, Hierarchical Scheduling and Transportation Tree (HSTT). Built on the dual deep Q-network (DDQN) framework, HSTT leverages the hierarchical structure of scheduling decisions to reduce encoding complexity and improve learning efficiency. Additionally, a local attention mechanism is integrated into the tree search process to constrain the decision space and enhance policy accuracy. Experimental results on benchmark datasets demonstrate that HSTT significantly outperforms traditional dispatching rules, metaheuristic methods, and existing deep reinforcement learning approaches in terms of makespan, runtime, and generalization performance.