通过多目标深度强化学习对边缘微服务部署和路由进行联合优化

IF 4.7 2区 计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS
Menglan Hu;Hao Wang;Xiaohui Xu;Jianwen He;Yi Hu;Tianping Deng;Kai Peng
{"title":"通过多目标深度强化学习对边缘微服务部署和路由进行联合优化","authors":"Menglan Hu;Hao Wang;Xiaohui Xu;Jianwen He;Yi Hu;Tianping Deng;Kai Peng","doi":"10.1109/TNSM.2024.3443872","DOIUrl":null,"url":null,"abstract":"Edge computing technologies with container-based microservice architectures promise to provide stable and low-latency services for large-scale and complex edge applications. However, due to the limited CPU and storage resources in edge computing scenarios, the coarse-grained service deployment on edge nodes causes performance bottlenecks. In addition, the effective deployment of microservices is tightly correlated with request routing, but the current research ignores the joint optimization of multi-instance deployment and routing. In this paper, we first model the problem of jointly optimizing service deployment and routing in a dynamically changing environment with multi-edge network collaboration based on a queuing network analysis. Secondly, we design heuristic algorithms to scale microservice instances horizontally in dynamic user request states. In addition, we propose a reinforcement learning algorithm based on reward shaping (RSPPO) to minimize user waiting delay and edge network resource consumption. We also solve the microservice deployment and request routing problem for multi-edge collaboration to achieve load balancing among edge nodes. Finally, extensive experiments verify the significant and extensive effectiveness of our algorithm.","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"21 6","pages":"6364-6381"},"PeriodicalIF":4.7000,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Joint Optimization of Microservice Deployment and Routing in Edge via Multi-Objective Deep Reinforcement Learning\",\"authors\":\"Menglan Hu;Hao Wang;Xiaohui Xu;Jianwen He;Yi Hu;Tianping Deng;Kai Peng\",\"doi\":\"10.1109/TNSM.2024.3443872\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Edge computing technologies with container-based microservice architectures promise to provide stable and low-latency services for large-scale and complex edge applications. However, due to the limited CPU and storage resources in edge computing scenarios, the coarse-grained service deployment on edge nodes causes performance bottlenecks. In addition, the effective deployment of microservices is tightly correlated with request routing, but the current research ignores the joint optimization of multi-instance deployment and routing. In this paper, we first model the problem of jointly optimizing service deployment and routing in a dynamically changing environment with multi-edge network collaboration based on a queuing network analysis. Secondly, we design heuristic algorithms to scale microservice instances horizontally in dynamic user request states. In addition, we propose a reinforcement learning algorithm based on reward shaping (RSPPO) to minimize user waiting delay and edge network resource consumption. We also solve the microservice deployment and request routing problem for multi-edge collaboration to achieve load balancing among edge nodes. Finally, extensive experiments verify the significant and extensive effectiveness of our algorithm.\",\"PeriodicalId\":13423,\"journal\":{\"name\":\"IEEE Transactions on Network and Service Management\",\"volume\":\"21 6\",\"pages\":\"6364-6381\"},\"PeriodicalIF\":4.7000,\"publicationDate\":\"2024-08-19\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Network and Service Management\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10639475/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Network and Service Management","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10639475/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

基于容器的微服务架构的边缘计算技术有望为大规模和复杂的边缘应用程序提供稳定和低延迟的服务。但是,由于边缘计算场景下CPU和存储资源有限,粗粒度业务部署在边缘节点上会造成性能瓶颈。此外,微服务的有效部署与请求路由密切相关,但目前的研究忽略了多实例部署和路由的联合优化。本文首先基于排队网络分析,对动态变化环境下多边缘网络协同下的业务部署和路由联合优化问题进行了建模。其次,我们设计了启发式算法来横向扩展动态用户请求状态下的微服务实例。此外,我们提出了一种基于奖励塑造(RSPPO)的强化学习算法,以最大限度地减少用户等待延迟和边缘网络资源消耗。解决了多边缘协作的微服务部署和请求路由问题,实现了边缘节点间的负载均衡。最后,通过大量的实验验证了该算法的显著和广泛的有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Joint Optimization of Microservice Deployment and Routing in Edge via Multi-Objective Deep Reinforcement Learning
Edge computing technologies with container-based microservice architectures promise to provide stable and low-latency services for large-scale and complex edge applications. However, due to the limited CPU and storage resources in edge computing scenarios, the coarse-grained service deployment on edge nodes causes performance bottlenecks. In addition, the effective deployment of microservices is tightly correlated with request routing, but the current research ignores the joint optimization of multi-instance deployment and routing. In this paper, we first model the problem of jointly optimizing service deployment and routing in a dynamically changing environment with multi-edge network collaboration based on a queuing network analysis. Secondly, we design heuristic algorithms to scale microservice instances horizontally in dynamic user request states. In addition, we propose a reinforcement learning algorithm based on reward shaping (RSPPO) to minimize user waiting delay and edge network resource consumption. We also solve the microservice deployment and request routing problem for multi-edge collaboration to achieve load balancing among edge nodes. Finally, extensive experiments verify the significant and extensive effectiveness of our algorithm.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE Transactions on Network and Service Management
IEEE Transactions on Network and Service Management Computer Science-Computer Networks and Communications
CiteScore
9.30
自引率
15.10%
发文量
325
期刊介绍: IEEE Transactions on Network and Service Management will publish (online only) peerreviewed archival quality papers that advance the state-of-the-art and practical applications of network and service management. Theoretical research contributions (presenting new concepts and techniques) and applied contributions (reporting on experiences and experiments with actual systems) will be encouraged. These transactions will focus on the key technical issues related to: Management Models, Architectures and Frameworks; Service Provisioning, Reliability and Quality Assurance; Management Functions; Enabling Technologies; Information and Communication Models; Policies; Applications and Case Studies; Emerging Technologies and Standards.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信