Graph reinforcement learning based dynamic scheduling for human-machine symbiosis manufacturing considering multi-type disturbance events

Abstract

Human-machine symbiosis manufacturing (HMSM) is widely used in aviation, aerospace, and marine industries due to its powerful production capacity. However, lots of manufacturing resources and multi-type disturbance events bring high complexity and strong uncertainty, which makes scheduling difficult. Meanwhile, deep reinforcement learning (DRL) is a promising information-driven decision-making technology. Therefore, this paper proposes a novel graph reinforcement learning method for the dynamic scheduling problem of HMSM. Specifically, a Markov decision process is established, in which the environment transition mechanism uses four key time points to solve the rarely studied constraints: calendar, normal commuting, and three-shifts. Then, a novel hierarchical aggregation graph neural network is designed, in which the subgraph cutting technology based on node type reduces the difficulty of graph calculation, and an aggregation architecture based on subgraph importance and attention mechanism is designed to achieve effective fusion of heterogeneous node information. Finally, a DRL algorithm with end-to-end action space is proposed, and a response mechanism for nine disturbance events is designed based on the state-action decision-making logic of DRL. Experimental results show that the proposed method outperforms scheduling rule, genetic programming, and two popular DRL methods, and can maintain stable production under uncertain environments.