Flexible Manufacturing Systems intralogistics: Dynamic optimization of AGVs and tool sharing using Colored-Timed Petri Nets and actor–critic RL with actions masking

IF 12.2 1区 工程技术 Q1 ENGINEERING, INDUSTRIAL
Sofiene Lassoued , Laxmikant Shrikant Baheti , Nathalie Weiß-Borkowski , Stefan Lier , Andreas Schwung
{"title":"Flexible Manufacturing Systems intralogistics: Dynamic optimization of AGVs and tool sharing using Colored-Timed Petri Nets and actor–critic RL with actions masking","authors":"Sofiene Lassoued ,&nbsp;Laxmikant Shrikant Baheti ,&nbsp;Nathalie Weiß-Borkowski ,&nbsp;Stefan Lier ,&nbsp;Andreas Schwung","doi":"10.1016/j.jmsy.2025.06.017","DOIUrl":null,"url":null,"abstract":"<div><div>Flexible Manufacturing Systems (FMS) are pivotal in optimizing production processes in today’s rapidly evolving manufacturing landscape. This paper advances the traditional job shop scheduling problem by incorporating additional complexities through the simultaneous integration of automated guided vehicles (AGVs) and tool-sharing systems. We propose a novel approach that combines Colored-Timed Petri Nets (CTPNs) with actor–critic model-based reinforcement learning (MBRL), effectively addressing the multifaceted challenges associated with FMS. CTPNs provide a formal modeling structure and dynamic action masking, significantly reducing the action search space, while MBRL ensures adaptability to changing environments through the learned policy. Leveraging the advantages of MBRL, we incorporate a lookahead strategy for optimal positioning of AGVs, improving operational efficiency. Our approach was evaluated on small-sized public benchmarks and a newly developed large-scale benchmark inspired by the Taillard benchmark. The results show that our approach matches traditional methods on smaller instances and outperforms them on larger ones in terms of makespan while achieving a tenfold reduction in computation time. To ensure reproducibility, we propose a gym-compatible environment and an instance generator. Additionally, an ablation study evaluates the contribution of each framework component to its overall performance.</div></div>","PeriodicalId":16227,"journal":{"name":"Journal of Manufacturing Systems","volume":"82 ","pages":"Pages 405-419"},"PeriodicalIF":12.2000,"publicationDate":"2025-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Manufacturing Systems","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0278612525001694","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, INDUSTRIAL","Score":null,"Total":0}
引用次数: 0

Abstract

Flexible Manufacturing Systems (FMS) are pivotal in optimizing production processes in today’s rapidly evolving manufacturing landscape. This paper advances the traditional job shop scheduling problem by incorporating additional complexities through the simultaneous integration of automated guided vehicles (AGVs) and tool-sharing systems. We propose a novel approach that combines Colored-Timed Petri Nets (CTPNs) with actor–critic model-based reinforcement learning (MBRL), effectively addressing the multifaceted challenges associated with FMS. CTPNs provide a formal modeling structure and dynamic action masking, significantly reducing the action search space, while MBRL ensures adaptability to changing environments through the learned policy. Leveraging the advantages of MBRL, we incorporate a lookahead strategy for optimal positioning of AGVs, improving operational efficiency. Our approach was evaluated on small-sized public benchmarks and a newly developed large-scale benchmark inspired by the Taillard benchmark. The results show that our approach matches traditional methods on smaller instances and outperforms them on larger ones in terms of makespan while achieving a tenfold reduction in computation time. To ensure reproducibility, we propose a gym-compatible environment and an instance generator. Additionally, an ablation study evaluates the contribution of each framework component to its overall performance.
柔性制造系统内部物流:使用有色时间Petri网和带有动作掩蔽的actor-critic RL的agv动态优化和工具共享
柔性制造系统(FMS)在优化当今快速发展的制造环境中的生产过程中起着关键作用。通过自动导引车(agv)和工具共享系统的同时集成,增加了作业车间调度的复杂性,从而对传统作业车间调度问题进行了改进。我们提出了一种将彩色时间Petri网(ctpn)与基于参与者-评论家模型的强化学习(MBRL)相结合的新方法,有效地解决了与FMS相关的多方面挑战。ctpn提供了形式化的建模结构和动态的动作掩蔽,大大减少了动作搜索空间,而MBRL通过学习策略确保了对不断变化的环境的适应性。利用MBRL的优势,我们结合了agv优化定位的前瞻性策略,提高了运营效率。我们的方法在小型公共基准和受tailard基准启发的新开发的大型基准上进行了评估。结果表明,我们的方法在较小的实例上与传统方法相匹配,在makespan方面在较大的实例上优于传统方法,同时将计算时间减少了十倍。为了确保再现性,我们提出了一个健身房兼容的环境和一个实例生成器。此外,消融研究评估了每个框架组件对其整体性能的贡献。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Manufacturing Systems
Journal of Manufacturing Systems 工程技术-工程:工业
CiteScore
23.30
自引率
13.20%
发文量
216
审稿时长
25 days
期刊介绍: The Journal of Manufacturing Systems is dedicated to showcasing cutting-edge fundamental and applied research in manufacturing at the systems level. Encompassing products, equipment, people, information, control, and support functions, manufacturing systems play a pivotal role in the economical and competitive development, production, delivery, and total lifecycle of products, meeting market and societal needs. With a commitment to publishing archival scholarly literature, the journal strives to advance the state of the art in manufacturing systems and foster innovation in crafting efficient, robust, and sustainable manufacturing systems. The focus extends from equipment-level considerations to the broader scope of the extended enterprise. The Journal welcomes research addressing challenges across various scales, including nano, micro, and macro-scale manufacturing, and spanning diverse sectors such as aerospace, automotive, energy, and medical device manufacturing.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信