{"title":"A method for noise-suppressed multimodal feature integration in urban scene detection","authors":"Xue-juan Han , Zhong Qu , Shu-fang Xia","doi":"10.1016/j.ipm.2025.104290","DOIUrl":null,"url":null,"abstract":"<div><div>The complementary imaging properties of visible and thermal infrared make them play a crucial role in multimodal object detection. Multimodal fusion methods that do not effectively deal with intra-modal and inter-modal noise interference can lead to degraded detection performance. To address this problem, we propose a generic multimodal object detection architecture. The noise within the input feature modality is first weakened by the Noise Suppression and Score-guided Fusion module (NSSFuse), while the intra-modal and inter-modal feature representations are enriched, thus facilitating the global interaction of multimodal features. Then the multimodal low-frequency features and high-frequency features are efficiently fused by the Multimodal Frequency Fusion module (MutiFreqFuse), which retains the key information while suppressing the inter-modal irrelevant noise to further enhance the multimodal feature fusion. Numerous experimental results validate the superiority of the model on the benchmark datasets, Multi-Modal Multi-Feature for Traffic Detection (M3FD) and Forward-Looking InfraRed (FLIR). The mean Average Precision (<em>mAP</em>) improves by 4.4–6.8% over the baseline models and is up to 6.3% higher than that of the most recent multimodal models.</div></div>","PeriodicalId":50365,"journal":{"name":"Information Processing & Management","volume":"62 6","pages":"Article 104290"},"PeriodicalIF":6.9000,"publicationDate":"2025-07-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information Processing & Management","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0306457325002316","RegionNum":1,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
The complementary imaging properties of visible and thermal infrared make them play a crucial role in multimodal object detection. Multimodal fusion methods that do not effectively deal with intra-modal and inter-modal noise interference can lead to degraded detection performance. To address this problem, we propose a generic multimodal object detection architecture. The noise within the input feature modality is first weakened by the Noise Suppression and Score-guided Fusion module (NSSFuse), while the intra-modal and inter-modal feature representations are enriched, thus facilitating the global interaction of multimodal features. Then the multimodal low-frequency features and high-frequency features are efficiently fused by the Multimodal Frequency Fusion module (MutiFreqFuse), which retains the key information while suppressing the inter-modal irrelevant noise to further enhance the multimodal feature fusion. Numerous experimental results validate the superiority of the model on the benchmark datasets, Multi-Modal Multi-Feature for Traffic Detection (M3FD) and Forward-Looking InfraRed (FLIR). The mean Average Precision (mAP) improves by 4.4–6.8% over the baseline models and is up to 6.3% higher than that of the most recent multimodal models.
期刊介绍:
Information Processing and Management is dedicated to publishing cutting-edge original research at the convergence of computing and information science. Our scope encompasses theory, methods, and applications across various domains, including advertising, business, health, information science, information technology marketing, and social computing.
We aim to cater to the interests of both primary researchers and practitioners by offering an effective platform for the timely dissemination of advanced and topical issues in this interdisciplinary field. The journal places particular emphasis on original research articles, research survey articles, research method articles, and articles addressing critical applications of research. Join us in advancing knowledge and innovation at the intersection of computing and information science.