基于多尺度空间传播和张量分解的深度补全

IF 2.6 4区 计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS
Mingming Sun, Tao Li, Qing Liao, Minghui Zhou
{"title":"基于多尺度空间传播和张量分解的深度补全","authors":"Mingming Sun,&nbsp;Tao Li,&nbsp;Qing Liao,&nbsp;Minghui Zhou","doi":"10.1016/j.jvcir.2025.104394","DOIUrl":null,"url":null,"abstract":"<div><div>Depth completion aims to generate dense depth maps from sparse depth maps. Existing approaches typically apply a spatial propagation module to iteratively refine depth values based on a single-scale initial depth map. In contrast, to overcome the limitations imposed by the convolution kernel size on propagation range, we propose a multi-scale spatial propagation module (MSSPM) that utilizes multi-scale features from the decoder to guide spatial propagation. To further enhance the model’s performance, we introduce a bottleneck feature enhancement module (BFEM) based on tensor decomposition, which can reduce feature redundancy and perform denoising through low-rank feature factorization. We also introduce a cross-layer feature fusion module (Fusion) to efficiently combine low-level encoder features and high-level decoder features. Extensive experiments on the indoor NYUv2 dataset and the outdoor KITTI dataset demonstrate the effectiveness of the proposed method.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"107 ","pages":"Article 104394"},"PeriodicalIF":2.6000,"publicationDate":"2025-01-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Depth completion based on multi-scale spatial propagation and tensor decomposition\",\"authors\":\"Mingming Sun,&nbsp;Tao Li,&nbsp;Qing Liao,&nbsp;Minghui Zhou\",\"doi\":\"10.1016/j.jvcir.2025.104394\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Depth completion aims to generate dense depth maps from sparse depth maps. Existing approaches typically apply a spatial propagation module to iteratively refine depth values based on a single-scale initial depth map. In contrast, to overcome the limitations imposed by the convolution kernel size on propagation range, we propose a multi-scale spatial propagation module (MSSPM) that utilizes multi-scale features from the decoder to guide spatial propagation. To further enhance the model’s performance, we introduce a bottleneck feature enhancement module (BFEM) based on tensor decomposition, which can reduce feature redundancy and perform denoising through low-rank feature factorization. We also introduce a cross-layer feature fusion module (Fusion) to efficiently combine low-level encoder features and high-level decoder features. Extensive experiments on the indoor NYUv2 dataset and the outdoor KITTI dataset demonstrate the effectiveness of the proposed method.</div></div>\",\"PeriodicalId\":54755,\"journal\":{\"name\":\"Journal of Visual Communication and Image Representation\",\"volume\":\"107 \",\"pages\":\"Article 104394\"},\"PeriodicalIF\":2.6000,\"publicationDate\":\"2025-01-31\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Visual Communication and Image Representation\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1047320325000082\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Visual Communication and Image Representation","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1047320325000082","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

深度补全旨在从稀疏深度图生成密集深度图。现有方法通常采用空间传播模块,在单比例尺初始深度图的基础上迭代细化深度值。相反,为了克服卷积核大小对传播范围的限制,我们提出了一种多尺度空间传播模块(MSSPM),该模块利用解码器的多尺度特征来指导空间传播。为了进一步提高模型的性能,我们引入了基于张量分解的瓶颈特征增强模块(BFEM),该模块可以减少特征冗余并通过低秩特征分解进行去噪。我们还引入了一个跨层特征融合模块(fusion),以有效地结合低级编码器特征和高级解码器特征。在室内NYUv2数据集和室外KITTI数据集上的大量实验证明了该方法的有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Depth completion based on multi-scale spatial propagation and tensor decomposition
Depth completion aims to generate dense depth maps from sparse depth maps. Existing approaches typically apply a spatial propagation module to iteratively refine depth values based on a single-scale initial depth map. In contrast, to overcome the limitations imposed by the convolution kernel size on propagation range, we propose a multi-scale spatial propagation module (MSSPM) that utilizes multi-scale features from the decoder to guide spatial propagation. To further enhance the model’s performance, we introduce a bottleneck feature enhancement module (BFEM) based on tensor decomposition, which can reduce feature redundancy and perform denoising through low-rank feature factorization. We also introduce a cross-layer feature fusion module (Fusion) to efficiently combine low-level encoder features and high-level decoder features. Extensive experiments on the indoor NYUv2 dataset and the outdoor KITTI dataset demonstrate the effectiveness of the proposed method.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Journal of Visual Communication and Image Representation
Journal of Visual Communication and Image Representation 工程技术-计算机:软件工程
CiteScore
5.40
自引率
11.50%
发文量
188
审稿时长
9.9 months
期刊介绍: The Journal of Visual Communication and Image Representation publishes papers on state-of-the-art visual communication and image representation, with emphasis on novel technologies and theoretical work in this multidisciplinary area of pure and applied research. The field of visual communication and image representation is considered in its broadest sense and covers both digital and analog aspects as well as processing and communication in biological visual systems.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信