Reinforcement learning vs rule-based dynamic movement strategies in UAV assisted networks

IF 5.8 2区 计算机科学 Q1 TELECOMMUNICATIONS
Adel Mounir Said , Michel Marot , Chérifa Boucetta , Hossam Afifi , Hassine Moungla , Gatien Roujanski
{"title":"Reinforcement learning vs rule-based dynamic movement strategies in UAV assisted networks","authors":"Adel Mounir Said ,&nbsp;Michel Marot ,&nbsp;Chérifa Boucetta ,&nbsp;Hossam Afifi ,&nbsp;Hassine Moungla ,&nbsp;Gatien Roujanski","doi":"10.1016/j.vehcom.2024.100788","DOIUrl":null,"url":null,"abstract":"<div><p>Since resource allocation of cellular networks is not dynamic, some cells may experience unplanned high traffic demands due to unexpected events. Unmanned aerial vehicles (UAV) can be used to provide the additional bandwidth required for data offloading.</p><p>Considering real-time and non-real-time traffic classes, our work is dedicated to optimize the placement of UAVs in cellular networks by two approaches. A first rule-based, low complexity method, that can be embedded in the UAV, while the other approach uses Reinforcement Learning (RL). It is based on Markov Decision Processes (MDP) for providing optimal results. The energy of the UAV battery and charging time constraints have been taken into account to cover a typical cellular environment consisting of many cells.</p><p>We used an open dataset for the Milan cellular network provided by Telecom Italia to evaluate the performance of both proposed models. Considering this dataset, the MDP model outperforms the rule-based algorithm. Nevertheless, the rule-based one requires less processing complexity and can be used immediately without any prior data. This work makes a notable contribution to developing practical and optimal solutions for UAV deployment in modern cellular networks.</p></div>","PeriodicalId":54346,"journal":{"name":"Vehicular Communications","volume":null,"pages":null},"PeriodicalIF":5.8000,"publicationDate":"2024-05-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Vehicular Communications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2214209624000639","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"TELECOMMUNICATIONS","Score":null,"Total":0}
引用次数: 0

Abstract

Since resource allocation of cellular networks is not dynamic, some cells may experience unplanned high traffic demands due to unexpected events. Unmanned aerial vehicles (UAV) can be used to provide the additional bandwidth required for data offloading.

Considering real-time and non-real-time traffic classes, our work is dedicated to optimize the placement of UAVs in cellular networks by two approaches. A first rule-based, low complexity method, that can be embedded in the UAV, while the other approach uses Reinforcement Learning (RL). It is based on Markov Decision Processes (MDP) for providing optimal results. The energy of the UAV battery and charging time constraints have been taken into account to cover a typical cellular environment consisting of many cells.

We used an open dataset for the Milan cellular network provided by Telecom Italia to evaluate the performance of both proposed models. Considering this dataset, the MDP model outperforms the rule-based algorithm. Nevertheless, the rule-based one requires less processing complexity and can be used immediately without any prior data. This work makes a notable contribution to developing practical and optimal solutions for UAV deployment in modern cellular networks.

无人机辅助网络中的强化学习与基于规则的动态运动策略对比
由于蜂窝网络的资源分配不是动态的,一些小区可能会因突发事件而出现计划外的高流量需求。考虑到实时和非实时流量等级,我们的工作致力于通过两种方法优化蜂窝网络中无人机的位置。第一种是基于规则的低复杂度方法,可嵌入无人机中;另一种方法则使用强化学习(RL)。它基于马尔可夫决策过程(MDP),可提供最佳结果。我们使用意大利电信公司提供的米兰蜂窝网络开放数据集来评估这两种模型的性能。考虑到该数据集,MDP 模型优于基于规则的算法。不过,基于规则的算法所需的处理复杂度较低,无需任何先验数据即可立即使用。这项工作为在现代蜂窝网络中部署无人机开发实用的最佳解决方案做出了显著贡献。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Vehicular Communications
Vehicular Communications Engineering-Electrical and Electronic Engineering
CiteScore
12.70
自引率
10.40%
发文量
88
审稿时长
62 days
期刊介绍: Vehicular communications is a growing area of communications between vehicles and including roadside communication infrastructure. Advances in wireless communications are making possible sharing of information through real time communications between vehicles and infrastructure. This has led to applications to increase safety of vehicles and communication between passengers and the Internet. Standardization efforts on vehicular communication are also underway to make vehicular transportation safer, greener and easier. The aim of the journal is to publish high quality peer–reviewed papers in the area of vehicular communications. The scope encompasses all types of communications involving vehicles, including vehicle–to–vehicle and vehicle–to–infrastructure. The scope includes (but not limited to) the following topics related to vehicular communications: Vehicle to vehicle and vehicle to infrastructure communications Channel modelling, modulating and coding Congestion Control and scalability issues Protocol design, testing and verification Routing in vehicular networks Security issues and countermeasures Deployment and field testing Reducing energy consumption and enhancing safety of vehicles Wireless in–car networks Data collection and dissemination methods Mobility and handover issues Safety and driver assistance applications UAV Underwater communications Autonomous cooperative driving Social networks Internet of vehicles Standardization of protocols.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信