Pedestrian detection based on YOLOv3 multimodal data fusion

IF 3.2 Q2 AUTOMATION & CONTROL SYSTEMS
Cheng Wang, Y. Liu, Fei-xiang Chang, Ming Lu
{"title":"Pedestrian detection based on YOLOv3 multimodal data fusion","authors":"Cheng Wang, Y. Liu, Fei-xiang Chang, Ming Lu","doi":"10.1080/21642583.2022.2129507","DOIUrl":null,"url":null,"abstract":"Multi-sensor fusion has essential applications in the field of target detection. Considering the current actual demand for miniaturization of on-board computers for driverless vehicles, this paper uses the multimodal data YOLOv3 (MDY) algorithm for pedestrian detection on embedded devices. The MDY algorithm uses YOLOv3 as the basic framework to improve pedestrian detection accuracy by optimizing anchor frames and adding small target detection branches. Then the algorithm is accelerated by using TensorRT technology to improve the real-time performance in embedded devices. Finally, a hybrid fusion framework is used to fuse the LIDAR point cloud data with the improved YOLOv3 algorithm to compensate for the shortcomings of a single sensor and improve the detection accuracy while ensuring speed. The improved YOLOv3 improves AP by 6.4% and speed by 11.3 FPS over the original algorithm. The MDY algorithm achieves better performance on the KITTI dataset. To further verify the feasibility of the MDY algorithm, an actual test was conducted on an unmanned vehicle with Jetson TX2 embedded device as the on-board computer within the campus scenario, and the results showed that the MDY algorithm achieves 90.8% accuracy under real-time operation and can achieve adequate detection accuracy and real-time performance on the embedded device.","PeriodicalId":46282,"journal":{"name":"Systems Science & Control Engineering","volume":null,"pages":null},"PeriodicalIF":3.2000,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Systems Science & Control Engineering","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/21642583.2022.2129507","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 4

Abstract

Multi-sensor fusion has essential applications in the field of target detection. Considering the current actual demand for miniaturization of on-board computers for driverless vehicles, this paper uses the multimodal data YOLOv3 (MDY) algorithm for pedestrian detection on embedded devices. The MDY algorithm uses YOLOv3 as the basic framework to improve pedestrian detection accuracy by optimizing anchor frames and adding small target detection branches. Then the algorithm is accelerated by using TensorRT technology to improve the real-time performance in embedded devices. Finally, a hybrid fusion framework is used to fuse the LIDAR point cloud data with the improved YOLOv3 algorithm to compensate for the shortcomings of a single sensor and improve the detection accuracy while ensuring speed. The improved YOLOv3 improves AP by 6.4% and speed by 11.3 FPS over the original algorithm. The MDY algorithm achieves better performance on the KITTI dataset. To further verify the feasibility of the MDY algorithm, an actual test was conducted on an unmanned vehicle with Jetson TX2 embedded device as the on-board computer within the campus scenario, and the results showed that the MDY algorithm achieves 90.8% accuracy under real-time operation and can achieve adequate detection accuracy and real-time performance on the embedded device.
基于YOLOv3多模态数据融合的行人检测
多传感器融合在目标检测领域有着重要的应用。考虑到当前无人驾驶汽车车载计算机小型化的实际需求,本文使用多模式数据YOLOv3(MDY)算法在嵌入式设备上进行行人检测。MDY算法以YOLOv3为基本框架,通过优化锚帧和添加小目标检测分支来提高行人检测精度。然后利用TensorRT技术对算法进行加速,以提高嵌入式设备的实时性。最后,采用混合融合框架,将改进的YOLOv3算法与激光雷达点云数据进行融合,以弥补单个传感器的不足,在保证速度的同时提高检测精度。改进后的YOLOv3比原来的算法提高了6.4%的AP和11.3FPS的速度。MDY算法在KITTI数据集上取得了较好的性能。为了进一步验证MDY算法的可行性,在校园场景下,以Jetson TX2嵌入式设备为车载计算机,在无人车上进行了实际测试,结果表明,MDY算法在实时操作下实现了90.8%的准确率,在嵌入式设备上能够实现足够的检测精度和实时性能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Systems Science & Control Engineering
Systems Science & Control Engineering AUTOMATION & CONTROL SYSTEMS-
CiteScore
9.50
自引率
2.40%
发文量
70
审稿时长
29 weeks
期刊介绍: Systems Science & Control Engineering is a world-leading fully open access journal covering all areas of theoretical and applied systems science and control engineering. The journal encourages the submission of original articles, reviews and short communications in areas including, but not limited to: · artificial intelligence · complex systems · complex networks · control theory · control applications · cybernetics · dynamical systems theory · operations research · systems biology · systems dynamics · systems ecology · systems engineering · systems psychology · systems theory
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信