Generative deep reinforcement learning method for dynamic parallel machines scheduling with adaptive maintenance activities

Abstract

In the process industries, where orders arrive at irregular intervals, inappropriate maintenance frequency often leads to unplanned shutdowns of high-speed parallel machines, resulting in unnecessary material consumption and a significant decline in the performance of the dynamic parallel machines scheduling. To address this issue, this paper proposes a generative deep reinforcement learning method that investigates the dynamic parallel machines scheduling problems with adaptive maintenance activities. Specifically, an enhanced Double DQN algorithm is proposed to schedule the dynamically arriving orders and maintenance activities, aiming to maximize average reliability while minimize the production costs. Additionally, a global exploration strategy is incorporated to enhance the scheduling and maintenance agent's global exploration capability, particularly in complex solution spaces with conflicting objectives. Furthermore, recognizing the difficulty of accurately capturing crucial scheduling and maintenance attributes within a predefined state space in a time-varying production environment, a guided Actor-Critic algorithm is introduced to autonomously generate the state space. Moreover, to tackle the unstable learning process caused by sparse rewards, a self-imitation learning is employed to guide the state space generation agent toward achieving rapid learning and convergence. Finally, simulation experiments validate that the proposed method not only autonomously enables state space generation but also exhibits superior performance for the investigated problem.