SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction.

Nivetha Jayakumar, Tonmoy Hossain, Miaomiao Zhang
{"title":"SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction.","authors":"Nivetha Jayakumar, Tonmoy Hossain, Miaomiao Zhang","doi":"10.1007/978-3-031-46914-5_23","DOIUrl":null,"url":null,"abstract":"<p><p>3D image reconstruction from a limited number of 2D images has been a long-standing challenge in computer vision and image analysis. While deep learning-based approaches have achieved impressive performance in this area, existing deep networks often fail to effectively utilize the shape structures of objects presented in images. As a result, the topology of reconstructed objects may not be well preserved, leading to the presence of artifacts such as discontinuities, holes, or mismatched connections between different parts. In this paper, we propose a shape-aware network based on diffusion models for 3D image reconstruction, named SADIR, to address these issues. In contrast to previous methods that primarily rely on spatial correlations of image intensities for 3D reconstruction, our model leverages shape priors learned from the training data to guide the reconstruction process. To achieve this, we develop a joint learning network that simultaneously learns a mean shape under deformation models. Each reconstructed image is then considered as a deformed variant of the mean shape. We validate our model, SADIR, on both brain and cardiac magnetic resonance images (MRIs). Experimental results show that our method outperforms the baselines with lower reconstruction error and better preservation of the shape structure of objects within the images.</p>","PeriodicalId":516531,"journal":{"name":"Shape in medical imaging : International Workshop, ShapeMI 2023, held in conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8, 2023, Proceedings. ShapeMI (Workshop) (2023 : Vancouver, B.C.)","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2023-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10977919/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Shape in medical imaging : International Workshop, ShapeMI 2023, held in conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8, 2023, Proceedings. ShapeMI (Workshop) (2023 : Vancouver, B.C.)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/978-3-031-46914-5_23","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2023/10/31 0:00:00","PubModel":"Epub","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

3D image reconstruction from a limited number of 2D images has been a long-standing challenge in computer vision and image analysis. While deep learning-based approaches have achieved impressive performance in this area, existing deep networks often fail to effectively utilize the shape structures of objects presented in images. As a result, the topology of reconstructed objects may not be well preserved, leading to the presence of artifacts such as discontinuities, holes, or mismatched connections between different parts. In this paper, we propose a shape-aware network based on diffusion models for 3D image reconstruction, named SADIR, to address these issues. In contrast to previous methods that primarily rely on spatial correlations of image intensities for 3D reconstruction, our model leverages shape priors learned from the training data to guide the reconstruction process. To achieve this, we develop a joint learning network that simultaneously learns a mean shape under deformation models. Each reconstructed image is then considered as a deformed variant of the mean shape. We validate our model, SADIR, on both brain and cardiac magnetic resonance images (MRIs). Experimental results show that our method outperforms the baselines with lower reconstruction error and better preservation of the shape structure of objects within the images.

SADIR:用于三维图像重建的形状感知扩散模型。
从数量有限的二维图像重建三维图像一直是计算机视觉和图像分析领域的一项长期挑战。虽然基于深度学习的方法在这一领域取得了令人瞩目的成绩,但现有的深度网络往往不能有效利用图像中物体的形状结构。因此,重建物体的拓扑结构可能无法得到很好的保留,从而导致不同部分之间出现不连续性、孔洞或不匹配连接等人工痕迹。为了解决这些问题,我们在本文中提出了一种基于扩散模型的形状感知网络,用于三维图像重建,并命名为 SADIR。与以往主要依靠图像强度的空间相关性进行三维重建的方法不同,我们的模型利用从训练数据中学到的形状先验来指导重建过程。为此,我们开发了一个联合学习网络,同时学习变形模型下的平均形状。然后,每个重建图像都被视为平均形状的变形变体。我们在脑部和心脏磁共振图像(MRI)上验证了我们的模型 SADIR。实验结果表明,我们的方法优于基线方法,重建误差更低,而且能更好地保留图像中物体的形状结构。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信