Ho-Gun Ha , Jinhan Lee , Gu-Hee Jung , Jaesung Hong , HyunKi Lee
{"title":"2D-3D Reconstruction of a Femur by Single X-Ray Image Based on Deep Transfer Learning Network","authors":"Ho-Gun Ha , Jinhan Lee , Gu-Hee Jung , Jaesung Hong , HyunKi Lee","doi":"10.1016/j.irbm.2024.100822","DOIUrl":null,"url":null,"abstract":"<div><h3>Objective</h3><p>Constructing a 3D model from its 2D images, known as 2D-3D reconstruction, is a challenging task. Conventionally, a parametric 3D model such as a statistical shape model (SSM) is deformed by matching the shapes in its 2D images through a series of processes, including calibration, 2D-3D registration, and optimization for nonrigid deformation. To overcome this complicated procedure, a streamlined 2D-3D reconstruction using a single X-ray image is developed in this study.</p></div><div><h3>Methods</h3><p>We propose 2D-3D reconstruction of a femur by adopting a deep neural network, where the deformation parameters in the SSM determining the 3D shape of the femur are predicted from a single X-ray image using a deep transfer-learning network. For learning the network from distinct features representing the 3D shape information in the X-ray image, a specific proximal part of the femur from a unique X-ray pose that allows accurate prediction of the 3D femur shape is designated and used to train the network. Then, the corresponding proximal/distal 3D femur model is reconstructed from only the single X-ray image acquired at the designated position.</p></div><div><h3>Results</h3><p><span>Experiments were conducted using actual X-ray images of a femur phantom and X-ray images of a patient's femur derived from computed tomography to verify the proposed method. The average errors of the reconstructed 3D shape of the proximal and </span>distal femurs from the proposed method were 1.20 mm and 1.08 mm in terms of root mean squared point-to-surface distance, respectively.</p></div><div><h3>Conclusion</h3><p>The proposed method presents an innovative approach to simplifying the 2D-3D reconstruction using deep neural networks that exhibits performance compatible with the existing methodologies.</p></div>","PeriodicalId":14605,"journal":{"name":"Irbm","volume":null,"pages":null},"PeriodicalIF":5.6000,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Irbm","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1959031824000034","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0
Abstract
Objective
Constructing a 3D model from its 2D images, known as 2D-3D reconstruction, is a challenging task. Conventionally, a parametric 3D model such as a statistical shape model (SSM) is deformed by matching the shapes in its 2D images through a series of processes, including calibration, 2D-3D registration, and optimization for nonrigid deformation. To overcome this complicated procedure, a streamlined 2D-3D reconstruction using a single X-ray image is developed in this study.
Methods
We propose 2D-3D reconstruction of a femur by adopting a deep neural network, where the deformation parameters in the SSM determining the 3D shape of the femur are predicted from a single X-ray image using a deep transfer-learning network. For learning the network from distinct features representing the 3D shape information in the X-ray image, a specific proximal part of the femur from a unique X-ray pose that allows accurate prediction of the 3D femur shape is designated and used to train the network. Then, the corresponding proximal/distal 3D femur model is reconstructed from only the single X-ray image acquired at the designated position.
Results
Experiments were conducted using actual X-ray images of a femur phantom and X-ray images of a patient's femur derived from computed tomography to verify the proposed method. The average errors of the reconstructed 3D shape of the proximal and distal femurs from the proposed method were 1.20 mm and 1.08 mm in terms of root mean squared point-to-surface distance, respectively.
Conclusion
The proposed method presents an innovative approach to simplifying the 2D-3D reconstruction using deep neural networks that exhibits performance compatible with the existing methodologies.
目标从二维图像构建三维模型(称为二维三维重建)是一项具有挑战性的任务。传统上,统计形状模型(SSM)等参数化三维模型是通过校准、二维三维配准和非刚性变形优化等一系列过程匹配其二维图像中的形状进行变形的。为了克服这一复杂的过程,本研究开发了一种使用单张 X 射线图像的简化 2D-3D 重建方法。我们建议采用深度神经网络对股骨进行 2D-3D 重建,其中,决定股骨 3D 形状的 SSM 中的变形参数将使用深度迁移学习网络从单张 X 射线图像中进行预测。为了从 X 射线图像中代表三维形状信息的不同特征中学习网络,需要指定能够准确预测股骨三维形状的独特 X 射线姿势中的特定股骨近端部分,并将其用于训练网络。然后,仅从指定位置获取的单张 X 光图像中重建相应的股骨近端/远端三维模型。结果实验使用股骨模型的实际 X 光图像和从计算机断层扫描中获取的患者股骨的 X 光图像来验证所提出的方法。根据所提方法重建的股骨近端和远端三维形状的平均误差(点到面距离的均方根值)分别为 1.20 毫米和 1.08 毫米。
期刊介绍:
IRBM is the journal of the AGBM (Alliance for engineering in Biology an Medicine / Alliance pour le génie biologique et médical) and the SFGBM (BioMedical Engineering French Society / Société française de génie biologique médical) and the AFIB (French Association of Biomedical Engineers / Association française des ingénieurs biomédicaux).
As a vehicle of information and knowledge in the field of biomedical technologies, IRBM is devoted to fundamental as well as clinical research. Biomedical engineering and use of new technologies are the cornerstones of IRBM, providing authors and users with the latest information. Its six issues per year propose reviews (state-of-the-art and current knowledge), original articles directed at fundamental research and articles focusing on biomedical engineering. All articles are submitted to peer reviewers acting as guarantors for IRBM''s scientific and medical content. The field covered by IRBM includes all the discipline of Biomedical engineering. Thereby, the type of papers published include those that cover the technological and methodological development in:
-Physiological and Biological Signal processing (EEG, MEG, ECG…)-
Medical Image processing-
Biomechanics-
Biomaterials-
Medical Physics-
Biophysics-
Physiological and Biological Sensors-
Information technologies in healthcare-
Disability research-
Computational physiology-
…