Machining parameter optimization for a batch milling system using multi-task deep reinforcement learning

IF 12.2 1区 工程技术 Q1 ENGINEERING, INDUSTRIAL
Pei Wang , Yixin Cui , Haizhen Tao , Xun Xu , Sheng Yang
{"title":"Machining parameter optimization for a batch milling system using multi-task deep reinforcement learning","authors":"Pei Wang ,&nbsp;Yixin Cui ,&nbsp;Haizhen Tao ,&nbsp;Xun Xu ,&nbsp;Sheng Yang","doi":"10.1016/j.jmsy.2024.11.013","DOIUrl":null,"url":null,"abstract":"<div><div>The integrated multi-objective optimization of machining parameters for improved machining quality and efficiency is important in batch milling systems. Due to the change of the batch milling system state, the continuous use of the same machining parameters may lead to degradation in quality and efficiency for workpieces in batches. Machining parameter optimization is usually determined by manual experience or trial-and-error methods, making it difficult to achieve a synergistic consideration of both quality and efficiency. To address this issue, a novel multi-task deep reinforcement learning method for machining parameter optimization in a batch machining system is proposed. Firstly, a reliable parallel joint estimation model of multiple machining quality and efficiency indicators is established using a multi-task time series estimation method, which can learn the correlation of these indicators to improve estimation accuracy. Then, the parameter optimization problem is formalized as a Markov decision process supported by a reinforcement learning virtual environment and an agent. The reinforcement learning virtual environment with the joint estimation model is constructed to improve the accuracy of optimized machining parameters for the collaborative optimization of quality and efficiency indicators. Within the virtual environment, time series sequential state, sequential action, multi-objective reward function, and constraint conditions adapted to the joint estimation model are defined to repeatedly evaluate different machining parameters. The agent with a multi-head attention and a dynamic weight adjustment mechanism is designed to improve the stability of the optimization process. Finally, experiments on a real machining dataset of thin-walled parts show that compared with the traditional deep reinforcement learning algorithm, the optimization effect of the proposed framework is improved by 9 %−12 %, and the standard deviation is decreased by 9 % −18 %.</div></div>","PeriodicalId":16227,"journal":{"name":"Journal of Manufacturing Systems","volume":"78 ","pages":"Pages 124-152"},"PeriodicalIF":12.2000,"publicationDate":"2024-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Manufacturing Systems","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S027861252400267X","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, INDUSTRIAL","Score":null,"Total":0}
引用次数: 0

Abstract

The integrated multi-objective optimization of machining parameters for improved machining quality and efficiency is important in batch milling systems. Due to the change of the batch milling system state, the continuous use of the same machining parameters may lead to degradation in quality and efficiency for workpieces in batches. Machining parameter optimization is usually determined by manual experience or trial-and-error methods, making it difficult to achieve a synergistic consideration of both quality and efficiency. To address this issue, a novel multi-task deep reinforcement learning method for machining parameter optimization in a batch machining system is proposed. Firstly, a reliable parallel joint estimation model of multiple machining quality and efficiency indicators is established using a multi-task time series estimation method, which can learn the correlation of these indicators to improve estimation accuracy. Then, the parameter optimization problem is formalized as a Markov decision process supported by a reinforcement learning virtual environment and an agent. The reinforcement learning virtual environment with the joint estimation model is constructed to improve the accuracy of optimized machining parameters for the collaborative optimization of quality and efficiency indicators. Within the virtual environment, time series sequential state, sequential action, multi-objective reward function, and constraint conditions adapted to the joint estimation model are defined to repeatedly evaluate different machining parameters. The agent with a multi-head attention and a dynamic weight adjustment mechanism is designed to improve the stability of the optimization process. Finally, experiments on a real machining dataset of thin-walled parts show that compared with the traditional deep reinforcement learning algorithm, the optimization effect of the proposed framework is improved by 9 %−12 %, and the standard deviation is decreased by 9 % −18 %.
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Manufacturing Systems
Journal of Manufacturing Systems 工程技术-工程:工业
CiteScore
23.30
自引率
13.20%
发文量
216
审稿时长
25 days
期刊介绍: The Journal of Manufacturing Systems is dedicated to showcasing cutting-edge fundamental and applied research in manufacturing at the systems level. Encompassing products, equipment, people, information, control, and support functions, manufacturing systems play a pivotal role in the economical and competitive development, production, delivery, and total lifecycle of products, meeting market and societal needs. With a commitment to publishing archival scholarly literature, the journal strives to advance the state of the art in manufacturing systems and foster innovation in crafting efficient, robust, and sustainable manufacturing systems. The focus extends from equipment-level considerations to the broader scope of the extended enterprise. The Journal welcomes research addressing challenges across various scales, including nano, micro, and macro-scale manufacturing, and spanning diverse sectors such as aerospace, automotive, energy, and medical device manufacturing.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信