4DRC-OC：在深度图辅助下的4D毫米波雷达相机在线校准

IF 4.6 2区计算机科学 Q2 ROBOTICS

IEEE Robotics and Automation Letters Pub Date : 2025-04-08 DOI:10.1109/LRA.2025.3558453

Long Zhuang;Yiqing Yao;Nuo Li;Zijian Wang;Lingtong Zhong;Zijing Zhang;Tao Zhang

{"title":"4DRC-OC：在深度图辅助下的4D毫米波雷达相机在线校准","authors":"Long Zhuang;Yiqing Yao;Nuo Li;Zijian Wang;Lingtong Zhong;Zijing Zhang;Tao Zhang","doi":"10.1109/LRA.2025.3558453","DOIUrl":null,"url":null,"abstract":"The online calibration of 4D millimeter-wave radar and camera is crucial for advancing perception and SLAM technologies in complex environments. It eliminates the reliance on manual labeling, offering real-time and convenience. However, the sparse nature of 4D radar point clouds presents challenges in establishing correspondences with camera images. This letter proposes an online 4D radar-camera online calibration method (4DRC-OC) that utilizes unified depth map representations for auxiliary training, ensuring feature alignment and modal unification between the two sensors. Due to the limited useful information within sparse depth maps, 4DRC-OC uses dynamic convolution to adaptively capture detailed features. Furthermore, this letter designs a correlation module based on channel-wise fusion (CMCF) that computes correlations between error depth maps and RGB-derived depth maps, thereby enhancing features to facilitate extrinsic parameter regression. Experimental results on the Dual-Radar dataset validate the superiority of the proposed approach in extrinsic calibration.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 6","pages":"5273-5280"},"PeriodicalIF":4.6000,"publicationDate":"2025-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"4DRC-OC: Online Calibration of 4D Millimeter Wave Radar-Camera With Depth Map Assistance\",\"authors\":\"Long Zhuang;Yiqing Yao;Nuo Li;Zijian Wang;Lingtong Zhong;Zijing Zhang;Tao Zhang\",\"doi\":\"10.1109/LRA.2025.3558453\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The online calibration of 4D millimeter-wave radar and camera is crucial for advancing perception and SLAM technologies in complex environments. It eliminates the reliance on manual labeling, offering real-time and convenience. However, the sparse nature of 4D radar point clouds presents challenges in establishing correspondences with camera images. This letter proposes an online 4D radar-camera online calibration method (4DRC-OC) that utilizes unified depth map representations for auxiliary training, ensuring feature alignment and modal unification between the two sensors. Due to the limited useful information within sparse depth maps, 4DRC-OC uses dynamic convolution to adaptively capture detailed features. Furthermore, this letter designs a correlation module based on channel-wise fusion (CMCF) that computes correlations between error depth maps and RGB-derived depth maps, thereby enhancing features to facilitate extrinsic parameter regression. Experimental results on the Dual-Radar dataset validate the superiority of the proposed approach in extrinsic calibration.\",\"PeriodicalId\":13241,\"journal\":{\"name\":\"IEEE Robotics and Automation Letters\",\"volume\":\"10 6\",\"pages\":\"5273-5280\"},\"PeriodicalIF\":4.6000,\"publicationDate\":\"2025-04-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Robotics and Automation Letters\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10950073/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ROBOTICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Robotics and Automation Letters","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10950073/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ROBOTICS","Score":null,"Total":0}

引用次数: 0

摘要

四维毫米波雷达和相机的在线校准对于推进复杂环境下的感知和SLAM技术至关重要。它消除了对人工标签的依赖，提供实时和方便。然而，四维雷达点云的稀疏特性在与相机图像建立对应关系方面提出了挑战。本文提出了一种在线4D雷达-相机在线校准方法（4DRC-OC），该方法利用统一的深度图表示进行辅助训练，确保两个传感器之间的特征对齐和模态统一。由于稀疏深度图中的有用信息有限，4DRC-OC使用动态卷积自适应捕获详细特征。此外，本文设计了一个基于信道融合（CMCF）的相关模块，该模块计算误差深度图和rgb衍生深度图之间的相关性，从而增强特征以促进外部参数回归。在双雷达数据集上的实验结果验证了该方法在外部标定中的优越性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

4DRC-OC: Online Calibration of 4D Millimeter Wave Radar-Camera With Depth Map Assistance

The online calibration of 4D millimeter-wave radar and camera is crucial for advancing perception and SLAM technologies in complex environments. It eliminates the reliance on manual labeling, offering real-time and convenience. However, the sparse nature of 4D radar point clouds presents challenges in establishing correspondences with camera images. This letter proposes an online 4D radar-camera online calibration method (4DRC-OC) that utilizes unified depth map representations for auxiliary training, ensuring feature alignment and modal unification between the two sensors. Due to the limited useful information within sparse depth maps, 4DRC-OC uses dynamic convolution to adaptively capture detailed features. Furthermore, this letter designs a correlation module based on channel-wise fusion (CMCF) that computes correlations between error depth maps and RGB-derived depth maps, thereby enhancing features to facilitate extrinsic parameter regression. Experimental results on the Dual-Radar dataset validate the superiority of the proposed approach in extrinsic calibration.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

IEEE Robotics and Automation Letters Computer Science-Computer Science Applications

CiteScore

9.60

自引率

15.40%

发文量

1428

期刊介绍： The scope of this journal is to publish peer-reviewed articles that provide a timely and concise account of innovative research ideas and application results, reporting significant theoretical findings and application case studies in areas of robotics and automation.