A Deep Reinforcement Learning Framework Assisted by Genetic Programming for Dynamic Flexible Job Shop Scheduling

Abstract

The dynamic flexible job shop scheduling problem with jobs arriving (DFJSP-JA) is a critical scheduling problem in electrolytic aluminum production processes within the aluminum industry. In the DFJSP-JA, job processing information is obtained after job arrival, thus requiring real-time decisions to minimize total tardiness (TTD). A tree-structured long short-term memory deep reinforcement learning framework with behavior clustering-based multitree genetic programming (TDRL-BCMTGP) is proposed to address the DFJSP-JA. Behavior clustering is introduced to group the candidates of the multitree genetic programming (MTGP) to expedite the iterative generation of high-quality scheduling rules by improving the population diversity of the MTGP. A tree-structured long short-term memory (Tree-LSTM) model trained via contrastive learning generates embedding vectors that capture the structural discrepancy and semantic similarity of the candidates. The embedding vectors are concatenated with the shop floor states and fed into the policy network to train an effective agent. Experimental results demonstrate that the TDRL-BCMTGP framework outperforms state-of-the-art methods in minimizing the TTD across four types of dynamic shop floor scenarios in electrolytic aluminum production processes, while maintaining robust generalization capability under simulated time delays and dynamic changes in shop floor machines.