A heuristic-assisted deep reinforcement learning algorithm for flexible job shop scheduling with transport constraints

Automated guided vehicles (AGVs) are widely used for transportation in flexible job shop (FJS) systems, and their transportation task scheduling has the same substantial impact on production efficiency as machine scheduling does. However, traditional FJS scheduling methods often prioritize job sequencing and machine selection while ignoring the impact of AGV transportation, resulting in suboptimal scheduling solutions and even difficulties in implementation. To address this issue, this paper formulates a cooperative scheduling model by introducing the AGV scheduling problem into the classical FJS scheduling problem, abbreviated as the FJS-AGV problem, with the objective of minimizing the makespan. With respect to the FJS-AGV problem, a heuristic-assisted deep Q-network (HA-DQN) algorithm is proposed, which leverages heuristic rules to enable the decision agent to perform multiple actions at each decision point, which includes determining the responses to the following questions: Which operation should be processed next? On which machine? By which AGV? This decision mechanism enables the agent to make more informed decisions, leading to improved performance and resource allocation in the FJS-AGV system. The practicability of the proposed FJS-AGV model and the efficiency of the HA-DQN algorithm in solving the FJS-AGV problem are verified through various international benchmarks. Specifically, when solving instances in a large benchmark, the HA-DQN algorithm yields a significant 12.63% reduction in makespan compared with that when traditional heuristics are employed.