Taming Event Cameras With Bio-Inspired Architecture and Algorithm: A Case for Drone Obstacle Avoidance

IF 7.7 2区计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS

IEEE Transactions on Mobile Computing Pub Date : 2024-12-23 DOI:10.1109/TMC.2024.3521044

Danyang Li;Jingao Xu;Zheng Yang;Yishujie Zhao;Hao Cao;Yunhao Liu;Longfei Shangguan

{"title":"Taming Event Cameras With Bio-Inspired Architecture and Algorithm: A Case for Drone Obstacle Avoidance","authors":"Danyang Li;Jingao Xu;Zheng Yang;Yishujie Zhao;Hao Cao;Yunhao Liu;Longfei Shangguan","doi":"10.1109/TMC.2024.3521044","DOIUrl":null,"url":null,"abstract":"Fast and accurate obstacle avoidance is crucial to drone safety. Yet existing on-board sensor modules such as frame cameras and radars are ill-suited for doing so due to their low temporal resolution or limited field of view. This paper presents <i>BioDrone</i>, a new design paradigm for drone obstacle avoidance using stereo event cameras. At the heart of BioDrone are three simple yet effective system designs inspired by the mammalian visual system, namely, a chiasm-inspired event filtering, a lateral geniculate nucleus (LGN)-inspired event matching, and a dorsal stream-inspired obstacle tracking. We implement BioDrone on FPGA through software-hardware co-design and deploy it on an industrial drone. In comparative experiments against two state-of-the-art event-based systems, BioDrone consistently achieves an obstacle detection rate of <inline-formula><tex-math>$> $</tex-math></inline-formula>90%, and an obstacle tracking error of <inline-formula><tex-math>$<$</tex-math></inline-formula>5.8 cm across all flight modes with an end-to-end latency of <inline-formula><tex-math>$<$</tex-math></inline-formula>6.4 ms, outperforming both baselines by over 44%.","PeriodicalId":50389,"journal":{"name":"IEEE Transactions on Mobile Computing","volume":"24 5","pages":"4202-4216"},"PeriodicalIF":7.7000,"publicationDate":"2024-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Mobile Computing","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10811856/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Fast and accurate obstacle avoidance is crucial to drone safety. Yet existing on-board sensor modules such as frame cameras and radars are ill-suited for doing so due to their low temporal resolution or limited field of view. This paper presents BioDrone, a new design paradigm for drone obstacle avoidance using stereo event cameras. At the heart of BioDrone are three simple yet effective system designs inspired by the mammalian visual system, namely, a chiasm-inspired event filtering, a lateral geniculate nucleus (LGN)-inspired event matching, and a dorsal stream-inspired obstacle tracking. We implement BioDrone on FPGA through software-hardware co-design and deploy it on an industrial drone. In comparative experiments against two state-of-the-art event-based systems, BioDrone consistently achieves an obstacle detection rate of

$> $

90%, and an obstacle tracking error of

$<$

5.8 cm across all flight modes with an end-to-end latency of

$<$

6.4 ms, outperforming both baselines by over 44%.

查看原文本刊更多论文

驯服事件相机与生物启发的架构和算法：无人机避障案例

快速准确的避障对无人机安全至关重要。然而，现有的车载传感器模块，如框架相机和雷达，由于它们的低时间分辨率或有限的视野，不适合这样做。本文介绍了BioDrone，一种利用立体事件相机进行无人机避障的新设计范例。BioDrone的核心是三个简单而有效的系统设计，它们的灵感来自哺乳动物的视觉系统，即交叉启发的事件过滤，侧向geniculate nucleus （LGN）启发的事件匹配，以及背侧流启发的障碍物跟踪。我们通过软硬件协同设计在FPGA上实现了BioDrone，并将其部署在工业无人机上。在与两种最先进的基于事件的系统的比较实验中，BioDrone始终实现了$>；在所有飞行模式下，障碍物跟踪误差为5.8 cm，端到端延迟为6.4 ms，比两个基线都高出44%以上。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Mobile Computing 工程技术-电信学

CiteScore

12.90

自引率

2.50%

发文量

403

审稿时长

6.6 months

期刊介绍： IEEE Transactions on Mobile Computing addresses key technical issues related to various aspects of mobile computing. This includes (a) architectures, (b) support services, (c) algorithm/protocol design and analysis, (d) mobile environments, (e) mobile communication systems, (f) applications, and (g) emerging technologies. Topics of interest span a wide range, covering aspects like mobile networks and hosts, mobility management, multimedia, operating system support, power management, online and mobile environments, security, scalability, reliability, and emerging technologies such as wearable computers, body area networks, and wireless sensor networks. The journal serves as a comprehensive platform for advancements in mobile computing research.