A deep reinforcement learning approach for dynamic job-shop scheduling problem considering time variable and new job arrivals

In recent years, the complexity of the production process due to increased demand for customization has greatly increased the difficulty of dynamic job-shop scheduling problem (DJSP). This paper proposes a deep reinforcement learning (DRL) approach to tackle the DJSP based on proximal policy optimization (PPO) algorithm. A novel state representation method that expresses state features as multi-channel images is proposed to simplify the state characterization process. Various heuristic-based priority dispatching rules (PDRs)are used to construct action space. By converting scheduling instances into images and leveraging the spatial pyramid pooling fast (SPPF) module for feature extraction, this model can handle scheduling instances of varying scales and map size-independent processing information matrix to fixed action space. Additionally, a dense reward based on a predefined scheduling region is developed to offer detailed guidance to the agent, enabling more precise and comprehensive policy assessment. Static tests are conducted on well-known benchmarks, and the experimental results indicate that our scheduling model surpasses the performance of the three latest DRL approaches on average. Compared with PDR methods, dynamic experiments demonstrate that the proposed DRL model excels in adaptability and robustness when new tasks arrive and the processing time fluctuates with uncertainty.