基于深度强化学习的固定翼无人机辅助移动中继网络运行轨迹路径设计

IF 5.8 2区 计算机科学 Q1 TELECOMMUNICATIONS
Tao Wang , Xiaodong Ji , Xuan Zhu , Cheng He , Jian-Feng Gu
{"title":"基于深度强化学习的固定翼无人机辅助移动中继网络运行轨迹路径设计","authors":"Tao Wang ,&nbsp;Xiaodong Ji ,&nbsp;Xuan Zhu ,&nbsp;Cheng He ,&nbsp;Jian-Feng Gu","doi":"10.1016/j.vehcom.2024.100851","DOIUrl":null,"url":null,"abstract":"<div><div>This paper studies a fixed-wing unmanned aerial vehicle (UAV) assisted mobile relaying network (FUAVMRN), where a fixed-wing UAV employs an out-band full-duplex relaying fashion to serve a ground source-destination pair. It is confirmed that for a FUAVMRN, straight path is not suitable for the case that a huge amount of data need to be delivered, while circular path may lead to low throughput if the distance of ground source-destination pair is large. Thus, a running-track path (RTP) design problem is investigated for the FUAVMRN with the goal of energy minimization. By dividing an RTP into two straight and two semicircular paths, the total energy consumption of the UAV and the total amount of data transferred from the ground source to the ground destination via the UAV relay are calculated. According to the framework of Deep Reinforcement Learning and taking the UAV's roll-angle limit into consideration, the RTP design problem is formulated as a Markov Decision Process problem, giving the state and action spaces in addition to the policy and reward functions. In order for the UAV relay to obtain the control policy, Deep Deterministic Policy Gradient (DDPG) is used to solve the path design problem, leading to a DDPG based algorithm for the RTP design. Computer simulations are performed and the results show that the DDPG based algorithm always converges when the number of training iterations is around 500, and compared with the circular and straight paths, the proposed RTP design can save at least 12.13 % of energy and 65.93 % of flight time when the ground source and the ground destination are located 2000 m apart and need to transfer <span><math><mrow><mn>5000</mn><mrow><mtext>bit</mtext><mo>/</mo><mtext>Hz</mtext></mrow></mrow></math></span> of data. Moreover, it is more practical and efficient in terms of energy saving compared with the Deep Q Network based design.</div></div>","PeriodicalId":54346,"journal":{"name":"Vehicular Communications","volume":"50 ","pages":"Article 100851"},"PeriodicalIF":5.8000,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Deep Reinforcement Learning based running-track path design for fixed-wing UAV assisted mobile relaying network\",\"authors\":\"Tao Wang ,&nbsp;Xiaodong Ji ,&nbsp;Xuan Zhu ,&nbsp;Cheng He ,&nbsp;Jian-Feng Gu\",\"doi\":\"10.1016/j.vehcom.2024.100851\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>This paper studies a fixed-wing unmanned aerial vehicle (UAV) assisted mobile relaying network (FUAVMRN), where a fixed-wing UAV employs an out-band full-duplex relaying fashion to serve a ground source-destination pair. It is confirmed that for a FUAVMRN, straight path is not suitable for the case that a huge amount of data need to be delivered, while circular path may lead to low throughput if the distance of ground source-destination pair is large. Thus, a running-track path (RTP) design problem is investigated for the FUAVMRN with the goal of energy minimization. By dividing an RTP into two straight and two semicircular paths, the total energy consumption of the UAV and the total amount of data transferred from the ground source to the ground destination via the UAV relay are calculated. According to the framework of Deep Reinforcement Learning and taking the UAV's roll-angle limit into consideration, the RTP design problem is formulated as a Markov Decision Process problem, giving the state and action spaces in addition to the policy and reward functions. In order for the UAV relay to obtain the control policy, Deep Deterministic Policy Gradient (DDPG) is used to solve the path design problem, leading to a DDPG based algorithm for the RTP design. Computer simulations are performed and the results show that the DDPG based algorithm always converges when the number of training iterations is around 500, and compared with the circular and straight paths, the proposed RTP design can save at least 12.13 % of energy and 65.93 % of flight time when the ground source and the ground destination are located 2000 m apart and need to transfer <span><math><mrow><mn>5000</mn><mrow><mtext>bit</mtext><mo>/</mo><mtext>Hz</mtext></mrow></mrow></math></span> of data. Moreover, it is more practical and efficient in terms of energy saving compared with the Deep Q Network based design.</div></div>\",\"PeriodicalId\":54346,\"journal\":{\"name\":\"Vehicular Communications\",\"volume\":\"50 \",\"pages\":\"Article 100851\"},\"PeriodicalIF\":5.8000,\"publicationDate\":\"2024-10-28\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Vehicular Communications\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S2214209624001268\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"TELECOMMUNICATIONS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Vehicular Communications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2214209624001268","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"TELECOMMUNICATIONS","Score":null,"Total":0}
引用次数: 0

摘要

本文研究了一种固定翼无人飞行器辅助移动中继网络(FUAVMRN),其中固定翼无人飞行器采用带外全双工中继方式为一对地面信源-信宿提供服务。研究证实,对于 FUAVMRN 来说,直线路径不适合需要传输大量数据的情况,而如果地面源-目的对的距离较远,圆形路径可能会导致吞吐量较低。因此,以能量最小化为目标,研究了 FUAVMRN 的运行轨迹路径(RTP)设计问题。通过将 RTP 划分为两条直线路径和两条半圆路径,计算出无人机的总能耗以及通过无人机中继从地面源传输到地面目的地的总数据量。根据深度强化学习的框架,并考虑到无人机的滚动角度限制,将 RTP 设计问题表述为马尔可夫决策过程问题,除了给出策略和奖励函数外,还给出了状态和行动空间。为了让无人机中继获得控制策略,使用了深度确定性策略梯度(DDPG)来解决路径设计问题,从而产生了一种基于 DDPG 的 RTP 设计算法。计算机仿真结果表明,当训练迭代次数为 500 次左右时,基于 DDPG 的算法总是收敛的;与圆形路径和直线路径相比,当地面信源和地面目的地相距 2000 m 且需要传输 5000bit/Hz 的数据时,所提出的 RTP 设计至少能节省 12.13% 的能量和 65.93% 的飞行时间。此外,与基于 Deep Q 网络的设计相比,该设计在节能方面更加实用和高效。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Deep Reinforcement Learning based running-track path design for fixed-wing UAV assisted mobile relaying network
This paper studies a fixed-wing unmanned aerial vehicle (UAV) assisted mobile relaying network (FUAVMRN), where a fixed-wing UAV employs an out-band full-duplex relaying fashion to serve a ground source-destination pair. It is confirmed that for a FUAVMRN, straight path is not suitable for the case that a huge amount of data need to be delivered, while circular path may lead to low throughput if the distance of ground source-destination pair is large. Thus, a running-track path (RTP) design problem is investigated for the FUAVMRN with the goal of energy minimization. By dividing an RTP into two straight and two semicircular paths, the total energy consumption of the UAV and the total amount of data transferred from the ground source to the ground destination via the UAV relay are calculated. According to the framework of Deep Reinforcement Learning and taking the UAV's roll-angle limit into consideration, the RTP design problem is formulated as a Markov Decision Process problem, giving the state and action spaces in addition to the policy and reward functions. In order for the UAV relay to obtain the control policy, Deep Deterministic Policy Gradient (DDPG) is used to solve the path design problem, leading to a DDPG based algorithm for the RTP design. Computer simulations are performed and the results show that the DDPG based algorithm always converges when the number of training iterations is around 500, and compared with the circular and straight paths, the proposed RTP design can save at least 12.13 % of energy and 65.93 % of flight time when the ground source and the ground destination are located 2000 m apart and need to transfer 5000bit/Hz of data. Moreover, it is more practical and efficient in terms of energy saving compared with the Deep Q Network based design.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Vehicular Communications
Vehicular Communications Engineering-Electrical and Electronic Engineering
CiteScore
12.70
自引率
10.40%
发文量
88
审稿时长
62 days
期刊介绍: Vehicular communications is a growing area of communications between vehicles and including roadside communication infrastructure. Advances in wireless communications are making possible sharing of information through real time communications between vehicles and infrastructure. This has led to applications to increase safety of vehicles and communication between passengers and the Internet. Standardization efforts on vehicular communication are also underway to make vehicular transportation safer, greener and easier. The aim of the journal is to publish high quality peer–reviewed papers in the area of vehicular communications. The scope encompasses all types of communications involving vehicles, including vehicle–to–vehicle and vehicle–to–infrastructure. The scope includes (but not limited to) the following topics related to vehicular communications: Vehicle to vehicle and vehicle to infrastructure communications Channel modelling, modulating and coding Congestion Control and scalability issues Protocol design, testing and verification Routing in vehicular networks Security issues and countermeasures Deployment and field testing Reducing energy consumption and enhancing safety of vehicles Wireless in–car networks Data collection and dissemination methods Mobility and handover issues Safety and driver assistance applications UAV Underwater communications Autonomous cooperative driving Social networks Internet of vehicles Standardization of protocols.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信