{"title":"Reinforcement learning based maintenance scheduling of flexible multi-machine manufacturing systems with varying interactive degradation","authors":"Jiangxi Chen, Xiaojun Zhou","doi":"10.1016/j.ress.2025.111018","DOIUrl":null,"url":null,"abstract":"<div><div>In flexible multi-machine manufacturing systems, variations in product types dynamically influence machine loads, subsequently affecting the degradation processes of the machines. Moreover, the interactive degradation between the upstream and downstream machines, caused by the product quality deviations, changes with the different production routes for the variable product types. These factors, combined with the uncertain production schedules, present significant challenges for effective maintenance scheduling. To address these challenges, the maintenance scheduling problem is modeled as a Hidden-Mode Markov Decision Process (HM-MDP), where product types are treated as hidden modes that influence machine degradation and the subsequent maintenance decisions. The Interactive Degradation-Aware Proximal Policy Optimization (IDAPPO) reinforcement learning framework is introduced, enhancing the PPO algorithm with Graph Neural Networks (GNNs) to capture interactive degradation among machines and Long Short-Term Memory (LSTM) networks to handle temporal variations in production schedules. An entropy-based exploration strategy further manages the uncertainty of production schedules, enabling IDAPPO to adaptively optimize maintenance actions. Extensive experiments on both small-scale (5-machine) and large-scale (24-machine) systems demonstrate significantly reduced system losses and accelerated convergence of IDAPPO compared to the baseline approaches. These results indicate that IDAPPO provides a scalable and adaptive solution for improving the efficiency and reliability of complex manufacturing environments.</div></div>","PeriodicalId":54500,"journal":{"name":"Reliability Engineering & System Safety","volume":"260 ","pages":"Article 111018"},"PeriodicalIF":9.4000,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Reliability Engineering & System Safety","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0951832025002194","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, INDUSTRIAL","Score":null,"Total":0}
引用次数: 0
Abstract
In flexible multi-machine manufacturing systems, variations in product types dynamically influence machine loads, subsequently affecting the degradation processes of the machines. Moreover, the interactive degradation between the upstream and downstream machines, caused by the product quality deviations, changes with the different production routes for the variable product types. These factors, combined with the uncertain production schedules, present significant challenges for effective maintenance scheduling. To address these challenges, the maintenance scheduling problem is modeled as a Hidden-Mode Markov Decision Process (HM-MDP), where product types are treated as hidden modes that influence machine degradation and the subsequent maintenance decisions. The Interactive Degradation-Aware Proximal Policy Optimization (IDAPPO) reinforcement learning framework is introduced, enhancing the PPO algorithm with Graph Neural Networks (GNNs) to capture interactive degradation among machines and Long Short-Term Memory (LSTM) networks to handle temporal variations in production schedules. An entropy-based exploration strategy further manages the uncertainty of production schedules, enabling IDAPPO to adaptively optimize maintenance actions. Extensive experiments on both small-scale (5-machine) and large-scale (24-machine) systems demonstrate significantly reduced system losses and accelerated convergence of IDAPPO compared to the baseline approaches. These results indicate that IDAPPO provides a scalable and adaptive solution for improving the efficiency and reliability of complex manufacturing environments.
期刊介绍:
Elsevier publishes Reliability Engineering & System Safety in association with the European Safety and Reliability Association and the Safety Engineering and Risk Analysis Division. The international journal is devoted to developing and applying methods to enhance the safety and reliability of complex technological systems, like nuclear power plants, chemical plants, hazardous waste facilities, space systems, offshore and maritime systems, transportation systems, constructed infrastructure, and manufacturing plants. The journal normally publishes only articles that involve the analysis of substantive problems related to the reliability of complex systems or present techniques and/or theoretical results that have a discernable relationship to the solution of such problems. An important aim is to balance academic material and practical applications.