Patchwise Trabecular Bone Reconstruction of a 2D Proximal Femur Using Deep Learning and Seamless Quilting Algorithm

Background and Objective

Current in vivo imaging modalities such as CT and MRI provide low-resolution (LR) skeletal images of a limited resolution (400 to 600 μm), which is insufficient to precisely evaluate bone strength. Similarly, recent deep learning technologies show a limitation in terms of upscale ratio and image size. They also require a large number of high-resolution (HR) reference images for training, which are unavailable to acquire in clinical practice. Although topology optimization shows the potential to reconstruct HR skeletal images from CT scan data, it requires extreme computing cost for a limited region of interest (ROI). The goal of this study is to acquire a 2D HR full proximal femur image by reconstructing HR patch images via a deep neural network and merging them seamlessly.

Methods

Topology optimization was conducted to generate synthetic proximal femur images. After these HR images were downscaled 10 times, finite element analysis was conducted to evaluate the structural behavior of the downscaled LR images. By dividing the proximal femur images into a set of patches which share their cut boundary, we could generate a total of 52,000 pairs of the HR and LR image patches and the LR structural behavior (nodal displacement in this study). Then, these patch-wise data were used to train three different deep neural networks: ResNet, U-Net, and SRGAN. Finally, after the HR patch images were upscaled 10 times by the trained networks, they were seamlessly merged by minimizing a structural discontinuity on the patch boundary.

Results

The reconstructed HR proximal femur images were evaluated at three different ROIs in terms of image quality, apparent stiffness, and trabecular morphometric indices. They showed characteristic trabecular patterns with no visible structural discontinuity between the patches in all ROIs. Among three networks, ResNet showed the best performance in all quantitative measures.

Conclusion

This study proposes a novel framework that incorporates deep learning-based patchwise reconstruction and seamless quilting algorithm. Because the proposed method requires a very small number of reference HR images (only 11 synthetic full proximal femur images in total), it could be expanded to reconstruct trabecular bone from 3D clinical CT scan data for more reliable bone strength assessment in clinical practice.