RGB-D显著缺陷检测的三分支跨模态交互网络

IF 5.9 2区 工程技术 Q1 ENGINEERING, ELECTRICAL & ELECTRONIC
Lisha Cui;Ming Ma;Chaochao Li;Xiaoheng Jiang;Zhiwen Song;Lijian Fan;Mingliang Xu
{"title":"RGB-D显著缺陷检测的三分支跨模态交互网络","authors":"Lisha Cui;Ming Ma;Chaochao Li;Xiaoheng Jiang;Zhiwen Song;Lijian Fan;Mingliang Xu","doi":"10.1109/TIM.2025.3599282","DOIUrl":null,"url":null,"abstract":"RGB defective images are abundant in color and texture, whereas depth images exhibit prominent defect shapes and boundaries. Given this fact, we propose a three-branch cross-modal interactive network for RGB-D salient defect detection (TCI-Net). Specifically, we first perform a real three-stream encoder–decoder network at both the image level and feature level, with each branch utilizing RGB, RGB-D, and depth images as input to fully extract the underlying complementary information from each modality. In the encoder stage, a differential guidance module (DGM) is proposed to guide the RGB branch in learning the boundary shape features of defects, while a fusion perception module (FPM) is devised to facilitate the depth branch in encoding more textual knowledge. In addition, we propose a cross-modal feature refinement module (CFRM) to bridge the feature gap between modalities and enhance information interaction. Finally, in the decoder stage, we incorporate boundary map supervision and a semantic guidance module (SGM) to enhance the details and contextual semantics of the defects, while gradually reconstructing the spatial scale. Extensive experiments and analyses on the RGB-D defect dataset NEU RSDDS-AUG demonstrate that the proposed TCI-Net significantly improves the segmentation accuracy compared with state-of-the-art algorithms.","PeriodicalId":13341,"journal":{"name":"IEEE Transactions on Instrumentation and Measurement","volume":"74 ","pages":"1-11"},"PeriodicalIF":5.9000,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A Three-Branch Cross-Modal Interactive Network for RGB-D Salient Defect Detection\",\"authors\":\"Lisha Cui;Ming Ma;Chaochao Li;Xiaoheng Jiang;Zhiwen Song;Lijian Fan;Mingliang Xu\",\"doi\":\"10.1109/TIM.2025.3599282\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"RGB defective images are abundant in color and texture, whereas depth images exhibit prominent defect shapes and boundaries. Given this fact, we propose a three-branch cross-modal interactive network for RGB-D salient defect detection (TCI-Net). Specifically, we first perform a real three-stream encoder–decoder network at both the image level and feature level, with each branch utilizing RGB, RGB-D, and depth images as input to fully extract the underlying complementary information from each modality. In the encoder stage, a differential guidance module (DGM) is proposed to guide the RGB branch in learning the boundary shape features of defects, while a fusion perception module (FPM) is devised to facilitate the depth branch in encoding more textual knowledge. In addition, we propose a cross-modal feature refinement module (CFRM) to bridge the feature gap between modalities and enhance information interaction. Finally, in the decoder stage, we incorporate boundary map supervision and a semantic guidance module (SGM) to enhance the details and contextual semantics of the defects, while gradually reconstructing the spatial scale. Extensive experiments and analyses on the RGB-D defect dataset NEU RSDDS-AUG demonstrate that the proposed TCI-Net significantly improves the segmentation accuracy compared with state-of-the-art algorithms.\",\"PeriodicalId\":13341,\"journal\":{\"name\":\"IEEE Transactions on Instrumentation and Measurement\",\"volume\":\"74 \",\"pages\":\"1-11\"},\"PeriodicalIF\":5.9000,\"publicationDate\":\"2025-09-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Instrumentation and Measurement\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/11146450/\",\"RegionNum\":2,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Instrumentation and Measurement","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/11146450/","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

摘要

RGB缺陷图像具有丰富的色彩和纹理,而深度图像具有突出的缺陷形状和边界。鉴于这一事实,我们提出了一个用于RGB-D显著缺陷检测的三分支跨模态交互网络(TCI-Net)。具体来说,我们首先在图像级和特征级执行一个真正的三流编码器-解码器网络,每个分支使用RGB, RGB- d和深度图像作为输入,以从每个模态中充分提取潜在的互补信息。在编码器阶段,提出了差分引导模块(DGM)来引导RGB分支学习缺陷的边界形状特征,设计了融合感知模块(FPM)来促进深度分支编码更多文本知识。此外,我们提出了一个跨模态特征细化模块(CFRM)来弥合模态之间的特征差距,增强信息交互。最后,在解码阶段,我们结合边界图监督和语义引导模块(SGM)来增强缺陷的细节和上下文语义,同时逐步重建空间尺度。在RGB-D缺陷数据集NEU RSDDS-AUG上的大量实验和分析表明,与现有算法相比,所提出的TCI-Net显著提高了分割精度。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
A Three-Branch Cross-Modal Interactive Network for RGB-D Salient Defect Detection
RGB defective images are abundant in color and texture, whereas depth images exhibit prominent defect shapes and boundaries. Given this fact, we propose a three-branch cross-modal interactive network for RGB-D salient defect detection (TCI-Net). Specifically, we first perform a real three-stream encoder–decoder network at both the image level and feature level, with each branch utilizing RGB, RGB-D, and depth images as input to fully extract the underlying complementary information from each modality. In the encoder stage, a differential guidance module (DGM) is proposed to guide the RGB branch in learning the boundary shape features of defects, while a fusion perception module (FPM) is devised to facilitate the depth branch in encoding more textual knowledge. In addition, we propose a cross-modal feature refinement module (CFRM) to bridge the feature gap between modalities and enhance information interaction. Finally, in the decoder stage, we incorporate boundary map supervision and a semantic guidance module (SGM) to enhance the details and contextual semantics of the defects, while gradually reconstructing the spatial scale. Extensive experiments and analyses on the RGB-D defect dataset NEU RSDDS-AUG demonstrate that the proposed TCI-Net significantly improves the segmentation accuracy compared with state-of-the-art algorithms.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE Transactions on Instrumentation and Measurement
IEEE Transactions on Instrumentation and Measurement 工程技术-工程:电子与电气
CiteScore
9.00
自引率
23.20%
发文量
1294
审稿时长
3.9 months
期刊介绍: Papers are sought that address innovative solutions to the development and use of electrical and electronic instruments and equipment to measure, monitor and/or record physical phenomena for the purpose of advancing measurement science, methods, functionality and applications. The scope of these papers may encompass: (1) theory, methodology, and practice of measurement; (2) design, development and evaluation of instrumentation and measurement systems and components used in generating, acquiring, conditioning and processing signals; (3) analysis, representation, display, and preservation of the information obtained from a set of measurements; and (4) scientific and technical support to establishment and maintenance of technical standards in the field of Instrumentation and Measurement.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信