GRFormer: 3D reconstruction of liver and tumor via gridding and transformer-based point cloud completion

Abstract

Computed Tomography (CT) images can provide detailed information about human organs and lesions. However, its two-dimensional (2D) representation lacks spatial three-dimensionality, making it difficult to visualize three-dimensional (3D) anatomical structures. Therefore reconstructing high-precision 3D shapes from 2D medical images has become a significant challenge in the field of computer vision and medical image analysis. To address this problem, we propose an innovative gridding and geometry-aware Transformer-based point cloud completion network (GRFormer) that can accurately reconstruct the 3D structure of liver and tumors based on 2D contour information. GRFormer adopts a dual-branch feature extractor design combined with a multi-stage point generation module, which achieves progressive reconstruction from coarse-grained to fine-grained. We conduct systematic experimental validation based on LiTS public dataset. The quantitative evaluation and qualitative visualization analysis jointly show that GRFormer is capable of high-fidelity reconstruction of liver and tumor 3D geometries. In addition, we validate the model on clinical data provided by Shandong Provincial Hospital, and the reconstruction results are highly consistent with the judgment of professional physicians, proving the validity and reliability of the model in the actual clinical environment. In cross-dataset tests, GRFormer demonstrates excellent generalization capabilities, providing reliable technical support for clinical diagnosis and treatment planning. The code is publicly available at:https://github.com/yuwenqian0606/GRFormer.