Multi-Modal Fusion Sensing: A Comprehensive Review of Millimeter-Wave Radar and Its Integration With Other Modalities

IF 34.4 1区计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS

IEEE Communications Surveys and Tutorials Pub Date : 2024-03-08 DOI:10.1109/COMST.2024.3398004

Shuai Wang;Luoyu Mei;Ruofeng Liu;Wenchao Jiang;Zhimeng Yin;Xianjun Deng;Tian He

{"title":"Multi-Modal Fusion Sensing: A Comprehensive Review of Millimeter-Wave Radar and Its Integration With Other Modalities","authors":"Shuai Wang;Luoyu Mei;Ruofeng Liu;Wenchao Jiang;Zhimeng Yin;Xianjun Deng;Tian He","doi":"10.1109/COMST.2024.3398004","DOIUrl":null,"url":null,"abstract":"Millimeter-wave (mmWave) radar, with its high resolution, sensitivity to micro-vibrations, and adaptability to various environmental conditions, holds immense potential across multi-modal fusion sensing. Although there exist review papers on mmWave radar, there is a noticeable lack of comprehensive reviews focusing on its multi-modal fusion sensing capabilities. Addressing this gap, our review offers an extensive exploration of mmWave radar multi-modal fusion sensing, emphasizing its integration with other modalities. This review discusses the complex realm of millimeter-wave radar multi-modal fusion sensing, detailing its importance, hardware and software aspects, principles, and current potential and applications. It delves into data characteristics and datasets associated with mmWave radar, focusing on Doppler, point cloud, and multi-modal data formats. The review highlights how these data types enhance multi-modal fusion sensing and discusses methodologies, including signal processing and learning algorithms. Three categories of multi-modal fusion methodologies are proposed to optimally manage and interpret fused data. Various practical applications of mmWave radar multi-modal fusion sensing are illustrated, underlining the unique capabilities it provides when integrated with other sensors. The review concludes by identifying potential future research avenues, underscoring the immense potential of this field for further exploration and advancement.","PeriodicalId":55029,"journal":{"name":"IEEE Communications Surveys and Tutorials","volume":"27 1","pages":"322-352"},"PeriodicalIF":34.4000,"publicationDate":"2024-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Communications Surveys and Tutorials","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10525189/","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Millimeter-wave (mmWave) radar, with its high resolution, sensitivity to micro-vibrations, and adaptability to various environmental conditions, holds immense potential across multi-modal fusion sensing. Although there exist review papers on mmWave radar, there is a noticeable lack of comprehensive reviews focusing on its multi-modal fusion sensing capabilities. Addressing this gap, our review offers an extensive exploration of mmWave radar multi-modal fusion sensing, emphasizing its integration with other modalities. This review discusses the complex realm of millimeter-wave radar multi-modal fusion sensing, detailing its importance, hardware and software aspects, principles, and current potential and applications. It delves into data characteristics and datasets associated with mmWave radar, focusing on Doppler, point cloud, and multi-modal data formats. The review highlights how these data types enhance multi-modal fusion sensing and discusses methodologies, including signal processing and learning algorithms. Three categories of multi-modal fusion methodologies are proposed to optimally manage and interpret fused data. Various practical applications of mmWave radar multi-modal fusion sensing are illustrated, underlining the unique capabilities it provides when integrated with other sensors. The review concludes by identifying potential future research avenues, underscoring the immense potential of this field for further exploration and advancement.

查看原文本刊更多论文

多模态融合传感：毫米波雷达及其与其他模式融合的全面回顾

毫米波（mmWave）雷达具有高分辨率、对微振动的敏感性和对各种环境条件的适应性，在多模态融合传感中具有巨大的潜力。虽然已有关于毫米波雷达的综述论文，但明显缺乏对其多模态融合传感能力的全面综述。为了解决这一差距，我们的综述提供了毫米波雷达多模态融合传感的广泛探索，强调其与其他模态的集成。本文讨论了毫米波雷达多模态融合传感的复杂领域，详细介绍了它的重要性、硬件和软件方面、原理以及目前的潜力和应用。它深入研究了与毫米波雷达相关的数据特征和数据集，重点是多普勒、点云和多模态数据格式。这篇综述强调了这些数据类型如何增强多模态融合传感，并讨论了方法，包括信号处理和学习算法。提出了三类多模态融合方法来优化管理和解释融合数据。说明了毫米波雷达多模态融合传感的各种实际应用，强调了它与其他传感器集成时提供的独特功能。本综述最后指出了潜在的未来研究途径，强调了该领域进一步探索和发展的巨大潜力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Communications Surveys and Tutorials COMPUTER SCIENCE, INFORMATION SYSTEMS-TELECOMMUNICATIONS

CiteScore

80.20

自引率

2.50%

发文量

审稿时长

6 months

期刊介绍： IEEE Communications Surveys & Tutorials is an online journal published by the IEEE Communications Society for tutorials and surveys covering all aspects of the communications field. Telecommunications technology is progressing at a rapid pace, and the IEEE Communications Society is committed to providing researchers and other professionals the information and tools to stay abreast. IEEE Communications Surveys and Tutorials focuses on integrating and adding understanding to the existing literature on communications, putting results in context. Whether searching for in-depth information about a familiar area or an introduction into a new area, IEEE Communications Surveys & Tutorials aims to be the premier source of peer-reviewed, comprehensive tutorials and surveys, and pointers to further sources. IEEE Communications Surveys & Tutorials publishes only articles exclusively written for IEEE Communications Surveys & Tutorials and go through a rigorous review process before their publication in the quarterly issues. A tutorial article in the IEEE Communications Surveys & Tutorials should be designed to help the reader to become familiar with and learn something specific about a chosen topic. In contrast, the term survey, as applied here, is defined to mean a survey of the literature. A survey article in IEEE Communications Surveys & Tutorials should provide a comprehensive review of developments in a selected area, covering its development from its inception to its current state and beyond, and illustrating its development through liberal citations from the literature. Both tutorials and surveys should be tutorial in nature and should be written in a style comprehensible to readers outside the specialty of the article.