Beyond Static Obstacles: Integrating Kalman Filter with Reinforcement Learning for Drone Navigation

IF 2.1 3区 工程技术 Q2 ENGINEERING, AEROSPACE
Francesco Marino, Giorgio Guglieri
{"title":"Beyond Static Obstacles: Integrating Kalman Filter with Reinforcement Learning for Drone Navigation","authors":"Francesco Marino, Giorgio Guglieri","doi":"10.3390/aerospace11050395","DOIUrl":null,"url":null,"abstract":"Autonomous drones offer immense potential in dynamic environments, but their navigation systems often struggle with moving obstacles. This paper presents a novel approach for drone trajectory planning in such scenarios, combining the Interactive Multiple Model (IMM) Kalman filter with Proximal Policy Optimization (PPO) reinforcement learning (RL). The IMM Kalman filter addresses state estimation challenges by modeling the potential motion patterns of moving objects. This enables accurate prediction of future object positions, even in uncertain environments. The PPO reinforcement learning algorithm then leverages these predictions to optimize the drone’s real-time trajectory. Additionally, the capability of PPO to work with continuous action spaces makes it ideal for the smooth control adjustments required for safe navigation. Our simulation results demonstrate the effectiveness of this combined approach. The drone successfully navigates complex dynamic environments, achieving collision avoidance and goal-oriented behavior. This work highlights the potential of integrating advanced state estimation and reinforcement learning techniques to enhance autonomous drone capabilities in unpredictable settings.","PeriodicalId":48525,"journal":{"name":"Aerospace","volume":null,"pages":null},"PeriodicalIF":2.1000,"publicationDate":"2024-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Aerospace","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.3390/aerospace11050395","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, AEROSPACE","Score":null,"Total":0}
引用次数: 0

Abstract

Autonomous drones offer immense potential in dynamic environments, but their navigation systems often struggle with moving obstacles. This paper presents a novel approach for drone trajectory planning in such scenarios, combining the Interactive Multiple Model (IMM) Kalman filter with Proximal Policy Optimization (PPO) reinforcement learning (RL). The IMM Kalman filter addresses state estimation challenges by modeling the potential motion patterns of moving objects. This enables accurate prediction of future object positions, even in uncertain environments. The PPO reinforcement learning algorithm then leverages these predictions to optimize the drone’s real-time trajectory. Additionally, the capability of PPO to work with continuous action spaces makes it ideal for the smooth control adjustments required for safe navigation. Our simulation results demonstrate the effectiveness of this combined approach. The drone successfully navigates complex dynamic environments, achieving collision avoidance and goal-oriented behavior. This work highlights the potential of integrating advanced state estimation and reinforcement learning techniques to enhance autonomous drone capabilities in unpredictable settings.
超越静态障碍:将卡尔曼滤波器与强化学习整合用于无人机导航
自主无人机在动态环境中具有巨大的潜力,但它们的导航系统往往在移动障碍物面前举步维艰。本文结合交互式多重模型(IMM)卡尔曼滤波器和近端策略优化(PPO)强化学习(RL),提出了在这种情况下进行无人机轨迹规划的新方法。IMM 卡尔曼滤波器通过对运动物体的潜在运动模式进行建模,解决了状态估计难题。这样,即使在不确定的环境中,也能准确预测未来物体的位置。然后,PPO 强化学习算法利用这些预测来优化无人机的实时轨迹。此外,PPO 能够处理连续的动作空间,因此非常适合安全导航所需的平滑控制调整。我们的模拟结果证明了这种组合方法的有效性。无人机成功地在复杂的动态环境中导航,实现了避免碰撞和以目标为导向的行为。这项工作凸显了整合先进状态估计和强化学习技术的潜力,以增强无人机在不可预测环境中的自主能力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Aerospace
Aerospace ENGINEERING, AEROSPACE-
CiteScore
3.40
自引率
23.10%
发文量
661
审稿时长
6 weeks
期刊介绍: Aerospace is a multidisciplinary science inviting submissions on, but not limited to, the following subject areas: aerodynamics computational fluid dynamics fluid-structure interaction flight mechanics plasmas research instrumentation test facilities environment material science structural analysis thermophysics and heat transfer thermal-structure interaction aeroacoustics optics electromagnetism and radar propulsion power generation and conversion fuels and propellants combustion multidisciplinary design optimization software engineering data analysis signal and image processing artificial intelligence aerospace vehicles'' operation, control and maintenance risk and reliability human factors human-automation interaction airline operations and management air traffic management airport design meteorology space exploration multi-physics interaction.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信