Elaheh Sabziyan Varnousfaderani, S. Shihab, E. F. Dulia
{"title":"深度调度:基于深度强化学习的先进空中交通车辆调度算法","authors":"Elaheh Sabziyan Varnousfaderani, S. Shihab, E. F. Dulia","doi":"10.2514/1.d0416","DOIUrl":null,"url":null,"abstract":"Near-future air taxi operations with electric vertical takeoff and landing aircraft will be constrained by the need for frequent recharging and limited takeoff and landing pads in vertiports and will be subject to time-varying demand and electricity prices, making the dispatch problem unique and particularly challenging to solve. Previously, the authors have developed optimization models to address this problem. Such optimization models, however, suffer from prohibitively high computational run times when the scale of the problem increases, making them less practical for real-world implementation. To overcome this issue, the authors have developed two deep reinforcement learning-based dispatch algorithms, namely, single-agent and multi-agent double dueling deep Q-network dispatch algorithms, where the objective is to maximize operating profit. A passenger transportation simulation environment was built to assess the performance of these algorithms across 36 numerical cases with varying numbers of vehicles and vertiports and amounts of demand. The results indicate that the multi-agent dispatch algorithm can closely approximate the optimal dispatch policy with significantly less computational expenses compared to the benchmark optimization model. The multi-agent algorithm was found to outperform the single-agent counterpart with respect to both profits generated and training time. Additionally, we implemented a heuristic-based algorithm, faster but less effective in generating profits compared to our two deep reinforcement learning-based algorithms.","PeriodicalId":36984,"journal":{"name":"Journal of Air Transportation","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"DeepDispatch: Deep Reinforcement Learning-Based Vehicle Dispatch Algorithm for Advanced Air Mobility\",\"authors\":\"Elaheh Sabziyan Varnousfaderani, S. Shihab, E. F. Dulia\",\"doi\":\"10.2514/1.d0416\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Near-future air taxi operations with electric vertical takeoff and landing aircraft will be constrained by the need for frequent recharging and limited takeoff and landing pads in vertiports and will be subject to time-varying demand and electricity prices, making the dispatch problem unique and particularly challenging to solve. Previously, the authors have developed optimization models to address this problem. Such optimization models, however, suffer from prohibitively high computational run times when the scale of the problem increases, making them less practical for real-world implementation. To overcome this issue, the authors have developed two deep reinforcement learning-based dispatch algorithms, namely, single-agent and multi-agent double dueling deep Q-network dispatch algorithms, where the objective is to maximize operating profit. A passenger transportation simulation environment was built to assess the performance of these algorithms across 36 numerical cases with varying numbers of vehicles and vertiports and amounts of demand. The results indicate that the multi-agent dispatch algorithm can closely approximate the optimal dispatch policy with significantly less computational expenses compared to the benchmark optimization model. The multi-agent algorithm was found to outperform the single-agent counterpart with respect to both profits generated and training time. Additionally, we implemented a heuristic-based algorithm, faster but less effective in generating profits compared to our two deep reinforcement learning-based algorithms.\",\"PeriodicalId\":36984,\"journal\":{\"name\":\"Journal of Air Transportation\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-06-10\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Air Transportation\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.2514/1.d0416\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"Social Sciences\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Air Transportation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.2514/1.d0416","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Social Sciences","Score":null,"Total":0}
DeepDispatch: Deep Reinforcement Learning-Based Vehicle Dispatch Algorithm for Advanced Air Mobility
Near-future air taxi operations with electric vertical takeoff and landing aircraft will be constrained by the need for frequent recharging and limited takeoff and landing pads in vertiports and will be subject to time-varying demand and electricity prices, making the dispatch problem unique and particularly challenging to solve. Previously, the authors have developed optimization models to address this problem. Such optimization models, however, suffer from prohibitively high computational run times when the scale of the problem increases, making them less practical for real-world implementation. To overcome this issue, the authors have developed two deep reinforcement learning-based dispatch algorithms, namely, single-agent and multi-agent double dueling deep Q-network dispatch algorithms, where the objective is to maximize operating profit. A passenger transportation simulation environment was built to assess the performance of these algorithms across 36 numerical cases with varying numbers of vehicles and vertiports and amounts of demand. The results indicate that the multi-agent dispatch algorithm can closely approximate the optimal dispatch policy with significantly less computational expenses compared to the benchmark optimization model. The multi-agent algorithm was found to outperform the single-agent counterpart with respect to both profits generated and training time. Additionally, we implemented a heuristic-based algorithm, faster but less effective in generating profits compared to our two deep reinforcement learning-based algorithms.