深度调度:基于深度强化学习的先进空中交通车辆调度算法

Q2 Social Sciences
Elaheh Sabziyan Varnousfaderani, S. Shihab, E. F. Dulia
{"title":"深度调度:基于深度强化学习的先进空中交通车辆调度算法","authors":"Elaheh Sabziyan Varnousfaderani, S. Shihab, E. F. Dulia","doi":"10.2514/1.d0416","DOIUrl":null,"url":null,"abstract":"Near-future air taxi operations with electric vertical takeoff and landing aircraft will be constrained by the need for frequent recharging and limited takeoff and landing pads in vertiports and will be subject to time-varying demand and electricity prices, making the dispatch problem unique and particularly challenging to solve. Previously, the authors have developed optimization models to address this problem. Such optimization models, however, suffer from prohibitively high computational run times when the scale of the problem increases, making them less practical for real-world implementation. To overcome this issue, the authors have developed two deep reinforcement learning-based dispatch algorithms, namely, single-agent and multi-agent double dueling deep Q-network dispatch algorithms, where the objective is to maximize operating profit. A passenger transportation simulation environment was built to assess the performance of these algorithms across 36 numerical cases with varying numbers of vehicles and vertiports and amounts of demand. The results indicate that the multi-agent dispatch algorithm can closely approximate the optimal dispatch policy with significantly less computational expenses compared to the benchmark optimization model. The multi-agent algorithm was found to outperform the single-agent counterpart with respect to both profits generated and training time. Additionally, we implemented a heuristic-based algorithm, faster but less effective in generating profits compared to our two deep reinforcement learning-based algorithms.","PeriodicalId":36984,"journal":{"name":"Journal of Air Transportation","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"DeepDispatch: Deep Reinforcement Learning-Based Vehicle Dispatch Algorithm for Advanced Air Mobility\",\"authors\":\"Elaheh Sabziyan Varnousfaderani, S. Shihab, E. F. Dulia\",\"doi\":\"10.2514/1.d0416\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Near-future air taxi operations with electric vertical takeoff and landing aircraft will be constrained by the need for frequent recharging and limited takeoff and landing pads in vertiports and will be subject to time-varying demand and electricity prices, making the dispatch problem unique and particularly challenging to solve. Previously, the authors have developed optimization models to address this problem. Such optimization models, however, suffer from prohibitively high computational run times when the scale of the problem increases, making them less practical for real-world implementation. To overcome this issue, the authors have developed two deep reinforcement learning-based dispatch algorithms, namely, single-agent and multi-agent double dueling deep Q-network dispatch algorithms, where the objective is to maximize operating profit. A passenger transportation simulation environment was built to assess the performance of these algorithms across 36 numerical cases with varying numbers of vehicles and vertiports and amounts of demand. The results indicate that the multi-agent dispatch algorithm can closely approximate the optimal dispatch policy with significantly less computational expenses compared to the benchmark optimization model. The multi-agent algorithm was found to outperform the single-agent counterpart with respect to both profits generated and training time. Additionally, we implemented a heuristic-based algorithm, faster but less effective in generating profits compared to our two deep reinforcement learning-based algorithms.\",\"PeriodicalId\":36984,\"journal\":{\"name\":\"Journal of Air Transportation\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-06-10\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Air Transportation\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.2514/1.d0416\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"Social Sciences\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Air Transportation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.2514/1.d0416","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Social Sciences","Score":null,"Total":0}
引用次数: 0

摘要

在不久的将来,电动垂直起降飞机的空中出租车运营将受到频繁充电的需求和机场有限的起降坪的限制,并将受到随时间变化的需求和电价的影响,这使得调度问题非常独特,解决起来特别具有挑战性。在此之前,作者已经开发了优化模型来解决这一问题。然而,当问题规模增大时,此类优化模型的计算运行时间过高,使其在实际应用中不那么实用。为了克服这一问题,作者开发了两种基于深度强化学习的调度算法,即单代理和多代理双决斗深度 Q 网络调度算法,其目标是实现运营利润最大化。作者建立了一个客运模拟环境,以评估这些算法在 36 个具有不同车辆数量、vertiports 和需求量的数字案例中的性能。结果表明,与基准优化模型相比,多代理调度算法能以更少的计算费用接近最优调度策略。我们发现,多代理算法在产生的利润和训练时间方面都优于单代理算法。此外,我们还实施了一种基于启发式的算法,与我们的两种基于深度强化学习的算法相比,该算法速度更快,但在产生利润方面效果较差。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
DeepDispatch: Deep Reinforcement Learning-Based Vehicle Dispatch Algorithm for Advanced Air Mobility
Near-future air taxi operations with electric vertical takeoff and landing aircraft will be constrained by the need for frequent recharging and limited takeoff and landing pads in vertiports and will be subject to time-varying demand and electricity prices, making the dispatch problem unique and particularly challenging to solve. Previously, the authors have developed optimization models to address this problem. Such optimization models, however, suffer from prohibitively high computational run times when the scale of the problem increases, making them less practical for real-world implementation. To overcome this issue, the authors have developed two deep reinforcement learning-based dispatch algorithms, namely, single-agent and multi-agent double dueling deep Q-network dispatch algorithms, where the objective is to maximize operating profit. A passenger transportation simulation environment was built to assess the performance of these algorithms across 36 numerical cases with varying numbers of vehicles and vertiports and amounts of demand. The results indicate that the multi-agent dispatch algorithm can closely approximate the optimal dispatch policy with significantly less computational expenses compared to the benchmark optimization model. The multi-agent algorithm was found to outperform the single-agent counterpart with respect to both profits generated and training time. Additionally, we implemented a heuristic-based algorithm, faster but less effective in generating profits compared to our two deep reinforcement learning-based algorithms.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Journal of Air Transportation
Journal of Air Transportation Social Sciences-Safety Research
CiteScore
2.80
自引率
0.00%
发文量
16
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信