An effective multi-agent-based graph reinforcement learning method for solving flexible job shop scheduling problem

Abstract

Flexible job shop scheduling problem (FJSP) is a complex optimization problem in intelligent manufacturing and plays a key role in improving productivity, which is characterized by that each operation can be processed by multiple machines. Most current research into FJSP focuses on finding a higher-quality scheduling scheme in a shorter time. However, existing studies are hard to optimize the operation sequencing and machine assignment strategies simultaneously, which is critical for making the optimal scheduling decision. Therefore, a multi-agent-based graph reinforcement learning (MAGRL) method is proposed to effectively solve FJSP. Firstly, the FJSP is modeled into two Markov decision processes (MDPs), where the operation and machine agents are adopted to control the operation sequencing and machine assignment respectively. Secondly, to effectively predict the operation sequencing and machine assignment strategies, an encoder-double-decoder architecture is designed, including an improved graph attention network (IGAT)-based encoder, an operation strategy network-based decoder, and a machine strategy network-based decoder. Thirdly, an automatic entropy adjustment multi-agent proximal policy optimization (AEA-MAPPO) algorithm is proposed for effectively training the operation and machine strategy networks to optimize the operation sequencing and machine assignment strategies simultaneously. Finally, the effectiveness of MAGRL is verified through experimental comparisons with the classical scheduling rules and state-of-the-art methods to solve FJSP. The results achieved on the randomly generated FJSP instances and two common benchmarks indicate that MAGRL can consume less solution time to achieve higher solution quality in solving different-sized FJSP instances, and the overall performance of MAGRL is superior to that of the comparison methods.