Dueling double deep Q-network-based stamping resources intelligent scheduling for automobile manufacturing in cloud manufacturing environment

Abstract

With the development of intelligent manufacturing, the automobile manufacturing industry has entered the"AI +"era, and the cloud manufacturing paradigm for the application of the automobile manufacturing industry is also in progress. As a key of automobile manufacturing, the stamping resources scheduling for automobile manufacturing (SRSAM) in the cloud manufacturing (CMfg) is characterized by unique domain-specific attributes concerning task architecture, the particularities of resource allocation, and the agility in transitioning between service types, which impedes the effective transference of classical manufacturing resource scheduling methodologies. Concurrently, the prevalent approaches to stamping scheduling concentrate predominantly on resources within the confines of stamping workshops and production lines, which are limited in scope. Such approaches are ill-suited for coping with the volatile and extensive resource landscape inherent to cloud manufacturing environments. To handle the above issues, this paper proposes to solve the SRSAM problem in CMfg with a novel scheduling model and intelligent scheduling method based on Dueling Double Deep Q-network (DDDQN). Firstly, we propose a stamping resource multi-objective scheduling model within the in-depth analysis of the SRSAM problem in CMfg and introduce a novel task structure to articulate the dependencies within the stamping tasks. Secondly, addressing the static and dynamic scheduling requirements, we construct a scheduling framework based on deep reinforcement learning, propose the strategy combination based on 5 resource selections and 12 task selections to generate Agent's actions. Finally, integrating the proposed scheduling framework and model, the DDDQN algorithm is designed to solve the optimal scheduling scheme. Experimental results indicate that the proposed method consistently matches or exceeds other DRL algorithms, including proximal policy optimization (PPO), Q-learning, Deep Q-network (DQN), Double DQN (DDQN), and Dueling DQN in terms of scheduling performance and model training.