基于双q学习的机会网络路由协议

IF 0.7 Q4 COMPUTER SCIENCE, INFORMATION SYSTEMS
Jagdeep Singh, S. K. Dhurandher, I. Woungang, L. Barolli
{"title":"基于双q学习的机会网络路由协议","authors":"Jagdeep Singh, S. K. Dhurandher, I. Woungang, L. Barolli","doi":"10.3233/jhs-222018","DOIUrl":null,"url":null,"abstract":"Opportunistic Delay Tolerant Networks also referred to as Opportunistic Networks (OppNets) are a subset of wireless networks having mobile nodes with discontinuous opportunistic connections. As such, developing a performant routing protocol in such an environment remains a challenge. Most research in the literature have shown that reinforcement learning-based routing algorithms can achieve a good routing performance, but these algorithms suffer from under-estimations and/or over-estimations. Toward addressing these shortcomings, in this paper, a Double Q-learning based routing protocol for Opportunistic Networks framework named Off-Policy Reinforcement-based Adaptive Learning (ORAL) is proposed, which selects the most suitable next-hop node to transmit the message toward its destination without any bias by using a weighted double Q-estimator. In the next-hop selection process, a probability-based reward mechanism is involved, which considers the node’s delivery probability and the frequency of encounters among the nodes to boost the protocol’s efficiency. Simulation results convey that the proposed ORAL protocol improves the message delivery ratio by maintaining a trade-off between underestimation and overestimation. Simulations are conducted using the HAGGLE INFOCOM 2006 real mobility data trace and synthetic model, showing that when time-to-live is varied, (1) the proposed ORAL scheme outperforms DQLR by 14.05%, 9.4%, 5.81% respectively in terms of delivery probability, overhead ratio and average delay; (2) it also outperforms RLPRoPHET by 16.17%, 9.2%, 6.85%, respectively in terms of delivery ratio, overhead ratio and average delay.","PeriodicalId":54809,"journal":{"name":"Journal of High Speed Networks","volume":"15 1","pages":"1-14"},"PeriodicalIF":0.7000,"publicationDate":"2022-11-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":"{\"title\":\"Double Q-learning based routing protocol for opportunistic networks\",\"authors\":\"Jagdeep Singh, S. K. Dhurandher, I. Woungang, L. Barolli\",\"doi\":\"10.3233/jhs-222018\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Opportunistic Delay Tolerant Networks also referred to as Opportunistic Networks (OppNets) are a subset of wireless networks having mobile nodes with discontinuous opportunistic connections. As such, developing a performant routing protocol in such an environment remains a challenge. Most research in the literature have shown that reinforcement learning-based routing algorithms can achieve a good routing performance, but these algorithms suffer from under-estimations and/or over-estimations. Toward addressing these shortcomings, in this paper, a Double Q-learning based routing protocol for Opportunistic Networks framework named Off-Policy Reinforcement-based Adaptive Learning (ORAL) is proposed, which selects the most suitable next-hop node to transmit the message toward its destination without any bias by using a weighted double Q-estimator. In the next-hop selection process, a probability-based reward mechanism is involved, which considers the node’s delivery probability and the frequency of encounters among the nodes to boost the protocol’s efficiency. Simulation results convey that the proposed ORAL protocol improves the message delivery ratio by maintaining a trade-off between underestimation and overestimation. Simulations are conducted using the HAGGLE INFOCOM 2006 real mobility data trace and synthetic model, showing that when time-to-live is varied, (1) the proposed ORAL scheme outperforms DQLR by 14.05%, 9.4%, 5.81% respectively in terms of delivery probability, overhead ratio and average delay; (2) it also outperforms RLPRoPHET by 16.17%, 9.2%, 6.85%, respectively in terms of delivery ratio, overhead ratio and average delay.\",\"PeriodicalId\":54809,\"journal\":{\"name\":\"Journal of High Speed Networks\",\"volume\":\"15 1\",\"pages\":\"1-14\"},\"PeriodicalIF\":0.7000,\"publicationDate\":\"2022-11-03\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"2\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of High Speed Networks\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.3233/jhs-222018\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of High Speed Networks","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3233/jhs-222018","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 2

摘要

机会容忍延迟网络也称为机会网络(OppNets),是无线网络的一个子集,具有不连续的机会连接的移动节点。因此,在这样的环境中开发高性能路由协议仍然是一个挑战。大多数文献研究表明,基于强化学习的路由算法可以获得良好的路由性能,但这些算法存在低估和/或高估的问题。针对这些不足,本文提出了一种基于双q学习的机会网络框架路由协议,即基于离线策略强化的自适应学习(ORAL),该协议使用加权双q估计器选择最合适的下一跳节点将消息无偏差地发送到目的地。在下一跳选择过程中,采用基于概率的奖励机制,考虑节点的投递概率和节点之间的相遇频率,提高协议的效率。仿真结果表明,所提出的ORAL协议通过保持低估和高估之间的平衡来提高消息传递率。利用HAGGLE INFOCOM 2006真实移动数据跟踪和综合模型进行了仿真,结果表明:当生存时间发生变化时,ORAL方案在交付概率、开销比和平均延迟方面分别优于DQLR方案14.05%、9.4%和5.81%;(2)在交付率、开销率和平均延迟方面,RLPRoPHET分别优于RLPRoPHET 16.17%、9.2%、6.85%。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Double Q-learning based routing protocol for opportunistic networks
Opportunistic Delay Tolerant Networks also referred to as Opportunistic Networks (OppNets) are a subset of wireless networks having mobile nodes with discontinuous opportunistic connections. As such, developing a performant routing protocol in such an environment remains a challenge. Most research in the literature have shown that reinforcement learning-based routing algorithms can achieve a good routing performance, but these algorithms suffer from under-estimations and/or over-estimations. Toward addressing these shortcomings, in this paper, a Double Q-learning based routing protocol for Opportunistic Networks framework named Off-Policy Reinforcement-based Adaptive Learning (ORAL) is proposed, which selects the most suitable next-hop node to transmit the message toward its destination without any bias by using a weighted double Q-estimator. In the next-hop selection process, a probability-based reward mechanism is involved, which considers the node’s delivery probability and the frequency of encounters among the nodes to boost the protocol’s efficiency. Simulation results convey that the proposed ORAL protocol improves the message delivery ratio by maintaining a trade-off between underestimation and overestimation. Simulations are conducted using the HAGGLE INFOCOM 2006 real mobility data trace and synthetic model, showing that when time-to-live is varied, (1) the proposed ORAL scheme outperforms DQLR by 14.05%, 9.4%, 5.81% respectively in terms of delivery probability, overhead ratio and average delay; (2) it also outperforms RLPRoPHET by 16.17%, 9.2%, 6.85%, respectively in terms of delivery ratio, overhead ratio and average delay.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Journal of High Speed Networks
Journal of High Speed Networks Computer Science-Computer Networks and Communications
CiteScore
1.80
自引率
11.10%
发文量
26
期刊介绍: The Journal of High Speed Networks is an international archival journal, active since 1992, providing a publication vehicle for covering a large number of topics of interest in the high performance networking and communication area. Its audience includes researchers, managers as well as network designers and operators. The main goal will be to provide timely dissemination of information and scientific knowledge. The journal will publish contributed papers on novel research, survey and position papers on topics of current interest, technical notes, and short communications to report progress on long-term projects. Submissions to the Journal will be refereed consistently with the review process of leading technical journals, based on originality, significance, quality, and clarity. The journal will publish papers on a number of topics ranging from design to practical experiences with operational high performance/speed networks.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信