AIVIO: Closed-Loop, Object-Relative Navigation of UAVs With AI-Aided Visual Inertial Odometry

IF 4.6 2区计算机科学 Q2 ROBOTICS

IEEE Robotics and Automation Letters Pub Date : 2024-10-14 DOI:10.1109/LRA.2024.3479713

Thomas Jantos;Martin Scheiber;Christian Brommer;Eren Allak;Stephan Weiss;Jan Steinbrener

{"title":"AIVIO: Closed-Loop, Object-Relative Navigation of UAVs With AI-Aided Visual Inertial Odometry","authors":"Thomas Jantos;Martin Scheiber;Christian Brommer;Eren Allak;Stephan Weiss;Jan Steinbrener","doi":"10.1109/LRA.2024.3479713","DOIUrl":null,"url":null,"abstract":"Object-relative mobile robot navigation is essential for a variety of tasks, e.g. autonomous critical infrastructure inspection, but requires the capability to extract semantic information about the objects of interest from raw sensory data. While deep learning-based (DL) methods excel at inferring semantic object information from images, such as class and relative 6\n<inline-formula><tex-math>$^{\\circ }$</tex-math></inline-formula>\n of freedom (6-DoF) pose, they are computationally demanding and thus often not suitable for payload constrained mobile robots. In this letter we present a real-time capable unmanned aerial vehicle (UAV) system for object-relative, closed-loop navigation with a minimal sensor configuration consisting of an inertial measurement unit (IMU) and RGB camera. Utilizing a DL-based object pose estimator, solely trained on synthetic data and optimized for companion board deployment, the object-relative pose measurements are fused with the IMU data to perform object-relative localization. We conduct multiple real-world experiments to validate the performance of our system for the challenging use case of power pole inspection. An example closed-loop flight is presented in the supplementary video.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"10764-10771"},"PeriodicalIF":4.6000,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10715655","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Robotics and Automation Letters","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10715655/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ROBOTICS","Score":null,"Total":0}

引用次数: 0

Abstract

Object-relative mobile robot navigation is essential for a variety of tasks, e.g. autonomous critical infrastructure inspection, but requires the capability to extract semantic information about the objects of interest from raw sensory data. While deep learning-based (DL) methods excel at inferring semantic object information from images, such as class and relative 6

$^{\circ }$

of freedom (6-DoF) pose, they are computationally demanding and thus often not suitable for payload constrained mobile robots. In this letter we present a real-time capable unmanned aerial vehicle (UAV) system for object-relative, closed-loop navigation with a minimal sensor configuration consisting of an inertial measurement unit (IMU) and RGB camera. Utilizing a DL-based object pose estimator, solely trained on synthetic data and optimized for companion board deployment, the object-relative pose measurements are fused with the IMU data to perform object-relative localization. We conduct multiple real-world experiments to validate the performance of our system for the challenging use case of power pole inspection. An example closed-loop flight is presented in the supplementary video.

查看原文本刊更多论文

AIVIO：利用人工智能辅助视觉惯性测距技术实现无人飞行器的闭环物体相关导航

物体相关移动机器人导航对于各种任务（例如关键基础设施的自主检查）至关重要，但需要具备从原始感知数据中提取相关物体语义信息的能力。虽然基于深度学习（DL）的方法擅长从图像中推断对象的语义信息，如类别和相对 6$^{\circ }$ 自由度（6-DoF）姿态，但它们对计算要求很高，因此通常不适合受有效载荷限制的移动机器人。在这封信中，我们介绍了一种具有实时能力的无人飞行器（UAV）系统，该系统采用由惯性测量单元（IMU）和 RGB 摄像头组成的最小传感器配置，可进行物体相关的闭环导航。利用基于 DL 的物体姿态估算器（该估算器仅在合成数据上进行了训练，并针对配套板的部署进行了优化），将物体相关姿态测量值与 IMU 数据融合，以执行物体相关定位。我们进行了多个真实世界实验，以验证我们的系统在电线杆检查这一具有挑战性的使用案例中的性能。补充视频中展示了一个闭环飞行示例。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Robotics and Automation Letters Computer Science-Computer Science Applications

CiteScore

9.60

自引率

15.40%

发文量

1428

期刊介绍： The scope of this journal is to publish peer-reviewed articles that provide a timely and concise account of innovative research ideas and application results, reporting significant theoretical findings and application case studies in areas of robotics and automation.