XRL-SHAP-Cache: an explainable reinforcement learning approach for intelligent edge service caching in content delivery networks

IF 7.3 2区 计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS
Xiaolong Xu, Fan Wu, Muhammad Bilal, Xiaoyu Xia, Wanchun Dou, Lina Yao, Weiyi Zhong
{"title":"XRL-SHAP-Cache: an explainable reinforcement learning approach for intelligent edge service caching in content delivery networks","authors":"Xiaolong Xu, Fan Wu, Muhammad Bilal, Xiaoyu Xia, Wanchun Dou, Lina Yao, Weiyi Zhong","doi":"10.1007/s11432-023-3987-y","DOIUrl":null,"url":null,"abstract":"<p>Content delivery networks (CDNs) play a pivotal role in the modern internet infrastructure by enabling efficient content delivery across diverse geographical regions. As an essential component of CDNs, the edge caching scheme directly influences the user experience by determining the caching and eviction of content on edge servers. With the emergence of 5G technology, traditional caching schemes have faced challenges in adapting to increasingly complex and dynamic network environments. Consequently, deep reinforcement learning (DRL) offers a promising solution for intelligent zero-touch network governance. However, the black-box nature of DRL models poses challenges in understanding and making trusting decisions. In this paper, we propose an explainable reinforcement learning (XRL)-based intelligent edge service caching approach, namely XRL-SHAP-Cache, which combines DRL with an explainable artificial intelligence (XAI) technique for cache management in CDNs. Instead of focusing solely on achieving performance gains, this study introduces a novel paradigm for providing interpretable caching strategies, thereby establishing a foundation for future transparent and trustworthy edge caching solutions. Specifically, a multi-level cache scheduling framework for CDNs was formulated theoretically, with the D3QN-based caching scheme serving as the targeted interpretable model. Subsequently, by integrating Deep-SHAP into our framework, the contribution of each state input feature to the agent’s Q-value output was calculated, thereby providing valuable insights into the decision-making process. The proposed XRL-SHAP-Cache approach was evaluated through extensive experiments to demonstrate the behavior of the scheduling agent in the face of different environmental inputs. The results demonstrate its strong explainability under various real-life scenarios while maintaining superior performance compared to traditional caching schemes in terms of cache hit ratio, quality of service (QoS), and space utilization.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"25 1","pages":""},"PeriodicalIF":7.3000,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Science China Information Sciences","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s11432-023-3987-y","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

Content delivery networks (CDNs) play a pivotal role in the modern internet infrastructure by enabling efficient content delivery across diverse geographical regions. As an essential component of CDNs, the edge caching scheme directly influences the user experience by determining the caching and eviction of content on edge servers. With the emergence of 5G technology, traditional caching schemes have faced challenges in adapting to increasingly complex and dynamic network environments. Consequently, deep reinforcement learning (DRL) offers a promising solution for intelligent zero-touch network governance. However, the black-box nature of DRL models poses challenges in understanding and making trusting decisions. In this paper, we propose an explainable reinforcement learning (XRL)-based intelligent edge service caching approach, namely XRL-SHAP-Cache, which combines DRL with an explainable artificial intelligence (XAI) technique for cache management in CDNs. Instead of focusing solely on achieving performance gains, this study introduces a novel paradigm for providing interpretable caching strategies, thereby establishing a foundation for future transparent and trustworthy edge caching solutions. Specifically, a multi-level cache scheduling framework for CDNs was formulated theoretically, with the D3QN-based caching scheme serving as the targeted interpretable model. Subsequently, by integrating Deep-SHAP into our framework, the contribution of each state input feature to the agent’s Q-value output was calculated, thereby providing valuable insights into the decision-making process. The proposed XRL-SHAP-Cache approach was evaluated through extensive experiments to demonstrate the behavior of the scheduling agent in the face of different environmental inputs. The results demonstrate its strong explainability under various real-life scenarios while maintaining superior performance compared to traditional caching schemes in terms of cache hit ratio, quality of service (QoS), and space utilization.

XRL-SHAP-Cache:用于内容交付网络中智能边缘服务缓存的可解释强化学习方法
内容分发网络(CDN)可在不同地理区域高效分发内容,在现代互联网基础设施中发挥着举足轻重的作用。作为 CDN 的重要组成部分,边缘缓存方案通过决定边缘服务器上内容的缓存和驱逐,直接影响用户体验。随着 5G 技术的出现,传统缓存方案在适应日益复杂多变的网络环境方面面临挑战。因此,深度强化学习(DRL)为智能零接触网络治理提供了一种前景广阔的解决方案。然而,DRL 模型的黑箱性质给理解和做出信任决策带来了挑战。在本文中,我们提出了一种基于可解释强化学习(XRL)的智能边缘服务缓存方法,即 XRL-SHAP-Cache,它将 DRL 与可解释人工智能(XAI)技术相结合,用于 CDN 中的缓存管理。本研究不仅关注性能提升,还引入了一种提供可解释缓存策略的新模式,从而为未来透明、可信的边缘缓存解决方案奠定了基础。具体来说,本研究从理论上提出了一个 CDN 多级缓存调度框架,并将基于 D3QN 的缓存方案作为目标可解释模型。随后,通过将 Deep-SHAP 集成到我们的框架中,计算了每个状态输入特征对代理 Q 值输出的贡献,从而为决策过程提供了有价值的见解。我们通过大量实验对所提出的 XRL-SHAP-Cache 方法进行了评估,以展示调度代理在面对不同环境输入时的行为。结果表明,与传统缓存方案相比,XRL-SHAP-Cache 在缓存命中率、服务质量(QoS)和空间利用率方面都保持了卓越的性能,同时在各种现实生活场景下都具有很强的可解释性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Science China Information Sciences
Science China Information Sciences COMPUTER SCIENCE, INFORMATION SYSTEMS-
CiteScore
12.60
自引率
5.70%
发文量
224
审稿时长
8.3 months
期刊介绍: Science China Information Sciences is a dedicated journal that showcases high-quality, original research across various domains of information sciences. It encompasses Computer Science & Technologies, Control Science & Engineering, Information & Communication Engineering, Microelectronics & Solid-State Electronics, and Quantum Information, providing a platform for the dissemination of significant contributions in these fields.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信