{"title":"A multi-agent reinforcement learning based scheduling strategy for flexible job shops under machine breakdowns","authors":"Lingling Lv, Jiaxin Fan, Chunjiang Zhang, Weiming Shen","doi":"10.1016/j.rcim.2024.102923","DOIUrl":null,"url":null,"abstract":"<div><div>In a highly disrupted workshop environment, machine failures may occur frequently, requiring real-time schedule repair strategies. This paper proposes a type-aware multi-agent deep reinforcement learning (MADRL) to address real-time schedule repair for the flexible job shop scheduling problem under machine breakdowns. First, the problem is modeled as a multi-agent Markov decision process. At each decision point, the relationships among machine agents and operations are represented using a heterogeneous graph. Based on the graph, machine node embeddings are obtained based on a meta-path type-aware (MPTA) recurrent neural network. Meanwhile, a heterogeneous graph attention network is introduced to aggregate the features of nodes in the heterogeneous graph, forming operation embeddings available for each machine agent’s selection. The type-aware process of obtaining machine node embeddings and operation embeddings utilizes a hypernetwork to achieve parameter adaptation of node types, edge types, and node locations. Finally, performing a cross-attention mechanism on machine node embedding and its candidate operation embeddings for selection. Compared with the heuristic rules and MADRL algorithms, numerical experiment results indicate that the proposed MADRL achieves the minimum stability objective while reducing the makespan of the schedule affected by machine breakdowns.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"93 ","pages":"Article 102923"},"PeriodicalIF":9.1000,"publicationDate":"2024-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Robotics and Computer-integrated Manufacturing","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0736584524002102","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0
Abstract
In a highly disrupted workshop environment, machine failures may occur frequently, requiring real-time schedule repair strategies. This paper proposes a type-aware multi-agent deep reinforcement learning (MADRL) to address real-time schedule repair for the flexible job shop scheduling problem under machine breakdowns. First, the problem is modeled as a multi-agent Markov decision process. At each decision point, the relationships among machine agents and operations are represented using a heterogeneous graph. Based on the graph, machine node embeddings are obtained based on a meta-path type-aware (MPTA) recurrent neural network. Meanwhile, a heterogeneous graph attention network is introduced to aggregate the features of nodes in the heterogeneous graph, forming operation embeddings available for each machine agent’s selection. The type-aware process of obtaining machine node embeddings and operation embeddings utilizes a hypernetwork to achieve parameter adaptation of node types, edge types, and node locations. Finally, performing a cross-attention mechanism on machine node embedding and its candidate operation embeddings for selection. Compared with the heuristic rules and MADRL algorithms, numerical experiment results indicate that the proposed MADRL achieves the minimum stability objective while reducing the makespan of the schedule affected by machine breakdowns.
期刊介绍:
The journal, Robotics and Computer-Integrated Manufacturing, focuses on sharing research applications that contribute to the development of new or enhanced robotics, manufacturing technologies, and innovative manufacturing strategies that are relevant to industry. Papers that combine theory and experimental validation are preferred, while review papers on current robotics and manufacturing issues are also considered. However, papers on traditional machining processes, modeling and simulation, supply chain management, and resource optimization are generally not within the scope of the journal, as there are more appropriate journals for these topics. Similarly, papers that are overly theoretical or mathematical will be directed to other suitable journals. The journal welcomes original papers in areas such as industrial robotics, human-robot collaboration in manufacturing, cloud-based manufacturing, cyber-physical production systems, big data analytics in manufacturing, smart mechatronics, machine learning, adaptive and sustainable manufacturing, and other fields involving unique manufacturing technologies.