Direct Camera-Only Bundle Adjustment for 3-D Textured Colon Surface Reconstruction Based on Pre-Operative Model

IF 3.4 Q2 ENGINEERING, BIOMEDICAL
Shuai Zhang;Liang Zhao;Shoudong Huang;Evangelos B. Mazomenos;Danail Stoyanov
{"title":"Direct Camera-Only Bundle Adjustment for 3-D Textured Colon Surface Reconstruction Based on Pre-Operative Model","authors":"Shuai Zhang;Liang Zhao;Shoudong Huang;Evangelos B. Mazomenos;Danail Stoyanov","doi":"10.1109/TMRB.2024.3517168","DOIUrl":null,"url":null,"abstract":"This paper addresses the problem of reconstructing textured colon surface maps using a sequence of monocular colonoscopic images together with a 3D colon mesh model that has been segmented in CT colonography. The problem is formulated as a direct bundle adjustment (BA) problem which simultaneously optimizes all camera poses and the intensity of vertices on the pre-operative mesh model. This optimization is achieved by maximizing photometric consistency among multiple views of 2D images and the pre-operative 3D mesh model. The key properties of our proposed direct BA formulation involve eliminating the need for reference image specification, data association (feature extraction and matching), and image depth information. Thus, the proposed method is particularly suitable for scenarios where distinct features and image depth are not available, such as 2D colonoscopic images. Furthermore, we have proven that solving the proposed direct BA using the Gauss-Newton (GN) algorithm has the merit of optimizing camera poses only, which is equivalent to optimizing camera poses and the intensities of 3D vertices on the mesh together. Thus, a direct camera-only BA algorithm is proposed and used for 3D textured colon reconstruction from textureless 2D colonoscopic images. Validations using simulation, phantom, and in-vivo datasets are performed to demonstrate the accuracy and feasibility of the proposed algorithm.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"7 1","pages":"242-253"},"PeriodicalIF":3.4000,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE transactions on medical robotics and bionics","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10798524/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0

Abstract

This paper addresses the problem of reconstructing textured colon surface maps using a sequence of monocular colonoscopic images together with a 3D colon mesh model that has been segmented in CT colonography. The problem is formulated as a direct bundle adjustment (BA) problem which simultaneously optimizes all camera poses and the intensity of vertices on the pre-operative mesh model. This optimization is achieved by maximizing photometric consistency among multiple views of 2D images and the pre-operative 3D mesh model. The key properties of our proposed direct BA formulation involve eliminating the need for reference image specification, data association (feature extraction and matching), and image depth information. Thus, the proposed method is particularly suitable for scenarios where distinct features and image depth are not available, such as 2D colonoscopic images. Furthermore, we have proven that solving the proposed direct BA using the Gauss-Newton (GN) algorithm has the merit of optimizing camera poses only, which is equivalent to optimizing camera poses and the intensities of 3D vertices on the mesh together. Thus, a direct camera-only BA algorithm is proposed and used for 3D textured colon reconstruction from textureless 2D colonoscopic images. Validations using simulation, phantom, and in-vivo datasets are performed to demonstrate the accuracy and feasibility of the proposed algorithm.
基于术前模型的三维纹理结肠表面重建的直接相机捆绑调整
本文解决了利用单眼结肠镜图像序列和CT结肠镜中分割的三维结肠网格模型重建结肠表面纹理图的问题。该问题被描述为一个直接束调整(BA)问题,该问题同时优化所有相机位姿和术前网格模型上顶点的强度。这种优化是通过最大化2D图像和术前3D网格模型的多个视图之间的光度一致性来实现的。我们提出的直接BA公式的关键特性包括消除对参考图像规范、数据关联(特征提取和匹配)和图像深度信息的需求。因此,所提出的方法特别适用于无法获得明显特征和图像深度的场景,例如2D结肠镜图像。此外,我们还证明了使用高斯-牛顿(GN)算法求解所提出的直接BA具有仅优化摄像机姿态的优点,这相当于同时优化摄像机姿态和网格上三维顶点的强度。因此,提出了一种直接的仅相机BA算法,并将其用于从无纹理的二维结肠镜图像中进行三维纹理结肠重建。使用模拟、模拟和体内数据集进行验证,以证明所提出算法的准确性和可行性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
6.80
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信