基于深度强化学习的5G切片网络在线准入控制和资源预留

IF 7.7 2区 计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS
Fang Li;Yijun Hao;Shusen Yang;Peng Zhao
{"title":"基于深度强化学习的5G切片网络在线准入控制和资源预留","authors":"Fang Li;Yijun Hao;Shusen Yang;Peng Zhao","doi":"10.1109/TMC.2025.3548767","DOIUrl":null,"url":null,"abstract":"Network slicing architecture is expected to fulfill network applications with heterogeneous requirements through efficient slice admission control (SAC) policies. Existing SAC approaches entirely rely on current limited observations to make admission decisions, ignoring the potential impact of future demands. The short-sighted behaviors lead to poor service performance and infrastructure providers’ (InPs’) revenue in practice. In this paper, we propose OACR<inline-formula><tex-math>$^{2}$</tex-math></inline-formula>, an online SAC approach based on deep reinforcement learning (DRL) that can exploit predictable future requests to make more precise admission control decisions for the long-term revenue, and reserve proper resources accordingly. Specifically, we design three novel schemes: (i) a requirement predictor based on long short-term memory (LSTM) and a novel input-output way to predict future unforeseen requests, (ii) a DRL admission controller based on the partially observable Markov decision process model to make precise admission decisions without accurate future request information, with the convergence strictly proved, and (iii) a decision defender to guarantee decision reliability. Extensive experiments on real-world traces demonstrate that compared to the No-wait, Wait-queue, and Wait-earliest time approaches, OACR<inline-formula><tex-math>$^{2}$</tex-math></inline-formula> improves InPs’ revenue and acceptance ratio by up to 40.9% and 16.7%, respectively, without sacrificing online inference time (within 0.9 milliseconds).","PeriodicalId":50389,"journal":{"name":"IEEE Transactions on Mobile Computing","volume":"24 8","pages":"7360-7376"},"PeriodicalIF":7.7000,"publicationDate":"2025-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"OACR$^{2}$2: Online Admission Control and Resource Reservation for 5G Slice Networks With Deep Reinforcement Learning\",\"authors\":\"Fang Li;Yijun Hao;Shusen Yang;Peng Zhao\",\"doi\":\"10.1109/TMC.2025.3548767\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Network slicing architecture is expected to fulfill network applications with heterogeneous requirements through efficient slice admission control (SAC) policies. Existing SAC approaches entirely rely on current limited observations to make admission decisions, ignoring the potential impact of future demands. The short-sighted behaviors lead to poor service performance and infrastructure providers’ (InPs’) revenue in practice. In this paper, we propose OACR<inline-formula><tex-math>$^{2}$</tex-math></inline-formula>, an online SAC approach based on deep reinforcement learning (DRL) that can exploit predictable future requests to make more precise admission control decisions for the long-term revenue, and reserve proper resources accordingly. Specifically, we design three novel schemes: (i) a requirement predictor based on long short-term memory (LSTM) and a novel input-output way to predict future unforeseen requests, (ii) a DRL admission controller based on the partially observable Markov decision process model to make precise admission decisions without accurate future request information, with the convergence strictly proved, and (iii) a decision defender to guarantee decision reliability. Extensive experiments on real-world traces demonstrate that compared to the No-wait, Wait-queue, and Wait-earliest time approaches, OACR<inline-formula><tex-math>$^{2}$</tex-math></inline-formula> improves InPs’ revenue and acceptance ratio by up to 40.9% and 16.7%, respectively, without sacrificing online inference time (within 0.9 milliseconds).\",\"PeriodicalId\":50389,\"journal\":{\"name\":\"IEEE Transactions on Mobile Computing\",\"volume\":\"24 8\",\"pages\":\"7360-7376\"},\"PeriodicalIF\":7.7000,\"publicationDate\":\"2025-03-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Mobile Computing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10915540/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Mobile Computing","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10915540/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

网络切片架构期望通过有效的片接纳控制(SAC)策略来满足具有异构需求的网络应用。现有的SAC方法完全依赖于当前有限的观察来做出录取决定,忽略了未来需求的潜在影响。这种短视的行为导致服务性能和基础设施提供商(inp)在实践中的收入下降。在本文中,我们提出了一种基于深度强化学习(DRL)的在线SAC方法OACR$^{2}$,该方法可以利用可预测的未来请求为长期收入做出更精确的准入控制决策,并相应地保留适当的资源。具体来说,我们设计了三种新颖的方案:(i)基于长短期记忆(LSTM)的需求预测器和一种新颖的输入输出方法来预测未来不可预见的请求;(ii)基于部分可观察马尔可夫决策过程模型的DRL准入控制器,在没有准确的未来请求信息的情况下做出精确的准入决策,并严格证明了收敛性;(iii)决策防御器来保证决策的可靠性。在现实世界中进行的大量实验表明,与No-wait, Wait-queue和Wait-earliest time方法相比,OACR$^{2}$分别将InPs的收入和接受率提高了40.9%和16.7%,而不会牺牲在线推理时间(在0.9毫秒内)。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
OACR$^{2}$2: Online Admission Control and Resource Reservation for 5G Slice Networks With Deep Reinforcement Learning
Network slicing architecture is expected to fulfill network applications with heterogeneous requirements through efficient slice admission control (SAC) policies. Existing SAC approaches entirely rely on current limited observations to make admission decisions, ignoring the potential impact of future demands. The short-sighted behaviors lead to poor service performance and infrastructure providers’ (InPs’) revenue in practice. In this paper, we propose OACR$^{2}$, an online SAC approach based on deep reinforcement learning (DRL) that can exploit predictable future requests to make more precise admission control decisions for the long-term revenue, and reserve proper resources accordingly. Specifically, we design three novel schemes: (i) a requirement predictor based on long short-term memory (LSTM) and a novel input-output way to predict future unforeseen requests, (ii) a DRL admission controller based on the partially observable Markov decision process model to make precise admission decisions without accurate future request information, with the convergence strictly proved, and (iii) a decision defender to guarantee decision reliability. Extensive experiments on real-world traces demonstrate that compared to the No-wait, Wait-queue, and Wait-earliest time approaches, OACR$^{2}$ improves InPs’ revenue and acceptance ratio by up to 40.9% and 16.7%, respectively, without sacrificing online inference time (within 0.9 milliseconds).
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE Transactions on Mobile Computing
IEEE Transactions on Mobile Computing 工程技术-电信学
CiteScore
12.90
自引率
2.50%
发文量
403
审稿时长
6.6 months
期刊介绍: IEEE Transactions on Mobile Computing addresses key technical issues related to various aspects of mobile computing. This includes (a) architectures, (b) support services, (c) algorithm/protocol design and analysis, (d) mobile environments, (e) mobile communication systems, (f) applications, and (g) emerging technologies. Topics of interest span a wide range, covering aspects like mobile networks and hosts, mobility management, multimedia, operating system support, power management, online and mobile environments, security, scalability, reliability, and emerging technologies such as wearable computers, body area networks, and wireless sensor networks. The journal serves as a comprehensive platform for advancements in mobile computing research.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信