Matrix manufacturing system layout and scheduling via graph neural network and multi‐action deep reinforcement learning

Abstract

Matrix manufacturing system (MMS) is a novel production paradigm in Industry 4.0 that provides a highly flexible production environment based on the principles of decentralization, modularity, and unrestricted connectivity. MMS effectively addresses the challenges of personalized customization, which, in turn, imposes stricter demands on the optimization of its layout and scheduling. However, existing research on MMS primarily focuses on constructing theoretical frameworks, with limited attention to practical layout and scheduling optimization. Moreover, layout and scheduling decisions in MMS are highly coupled, and the system state has complex topological structures and dynamics. Conventional vector representation methods struggle to fully capture these intricate relationships, which limits the ability of MMS to address complex production demands. Therefore, to solve the MMS layout and scheduling (MMSLS) problem, this paper proposes an end-to-end multi-action deep reinforcement learning (MADRL) method based on a three-stage embedded heterogeneous graph neural network (HGNN) to learn the optimal policy for parallel decision-making for MMSLS, which aims to minimize makespan. Firstly, the traditional disjunctive graph of flexible scheduling problems is expanded into a heterogeneous graph by incorporating workstation and location nodes, which more intuitively captures the complex associations in MMS between operations and workstations and between workstations and locations. Secondly, we propose a novel HGNN algorithm to enhance representation learning by first transforming MMSLS heterogeneous graph into node-level embeddings and then using heterogeneous graph-level representation vectors as inputs. Finally, the agent sequentially performs actions based on two parameterized sub-policies, operation-workstation actions and location actions, which are trained to learn the optimal MMSLS policy using the proximal policy optimization (PPO) algorithm. Experimental results from both randomized and benchmark instances reveal that the proposed method not only outperforms manually crafted heuristic scheduling rules in solution quality but also exceeds metaheuristic algorithms in computational velocity. Furthermore, it demonstrates strong generalization when handling larger-scale instances.