A novel deep self-learning method for flexible job-shop scheduling problems with multiplicity: Deep reinforcement learning assisted the fluid master-apprentice evolutionary algorithm

Abstract

In today’s dynamic environment, companies must navigate highly competitive markets. They consistently need to implement new technologies and deliver the right product at the right time in response to customer demand. This necessitates a high level of adaptability and efficiency in their manufacturing processes. Flexible job-shops offer a more efficient alternative to traditional manufacturing practices by accommodating these needs. Additionally, in actual manufacturing plants, multiple jobs are typically required for each part type. To address this complexity, this article investigates the flexible job-shop scheduling problem with multiplicity (MFJSP). We propose a deep self-learning method based on deep reinforcement learning and fluid master-apprentice evolutionary algorithm (DSLFMAE) to minimize makespan for the MFJSP. The proposed DSLFMAE is the integration of a fluid master-apprentice evolutionary (FMAE) algorithm and a proximal policy optimization (PPO) algorithm. The FMAE algorithm serves as the core optimization method, employing the PPO algorithm to dynamically adjust the control parameters of the FMAE algorithm during the optimization process. Twelve state features are extracted to capture the evolutionary states of the FMAE algorithm accurately, and a long short-term memory Q-network (LSTM-Q) is designed to encode these continuous states. Subsequently, to adjust multiple interrelated control parameters of the FMAE algorithm simultaneously, a multivariate Gaussian distribution-based PPO algorithm is developed to train the LSTM-Q network. Numerical outcomes show the efficacy and superiority of the DSLFMAE in addressing the flexible job-shop scheduling problem with multiplicity (MFJSP) across different scales.