Integrated dynamic scheduling method for hybrid flow shop with machine preventive maintenance based on cooperative multi-agent deep reinforcement learning

Hybrid flow shop widely used in manufacturing industry is facing the challenge of complex and dynamic production environment. Current study mostly cannot consider the machine preventive maintenance and dynamic events in hybrid flow shop scheduling process. Therefore, this paper presents an integrated dynamic scheduling method for hybrid flow shop scheduling problem- unrelated parallel machine considering preventive maintenance (DHSFP-UPM-PM). And the multi-scheduling objectives include minimizing completion time, processing cost and maintenance cost. Firstly, the definition of research problem and the basic maintenance strategy are presented in detail. And an integrated mathematic model of DHSFP-UPM-PM is constructed. Then the integrated dynamic scheduling framework based on cooperative multi-agent deep reinforcement learning is proposed to solve the DHSFP-UPM-PM. Based on the above, we proposed a cooperative multi-processing stage agents (PSA) approach to realize the transformation from traditional single-agent to multi-agent. Meanwhile, the cooperative multi-agent Markova Decision Process is formulated to clarify the interaction between each agent and production environment. The state and action space as the key elements of scheduling model is also designed for each PSA. To optimize scheduling objectives, this paper further formulates new global reward mechanism and centralized training-decentralized execution method based on multi agent proximal policy optimization. Lastly, the experiment results verify the superiority and effectiveness of the proposed method when solving integrated scheduling problem and dynamic event. And the proposed method presents remarkable adaptability and flexibility under a different production scenario which prove the benefits of multi-agent deep reinforcement learning in complex and dynamic environment.