A Multiscale Spatial Transformer U-Net for Simultaneously Automatic Reorientation and Segmentation of 3-D Nuclear Cardiac Images

Accurate reorientation and segmentation of the left ventricular (LV) is essential for the quantitative analysis of myocardial perfusion imaging (MPI). This study proposes an end-to-end model, named as multiscale spatial transformer UNet (MS-ST-UNet), which involves the multiscale spatial transformer network (MSSTN) and multiscale UNet (MSUNet) modules to perform simultaneous reorientation and segmentation of LV region from nuclear cardiac images. The multiscale sampler produces images with varying resolutions, while scale transformer (ST) blocks are employed to align the scales of features. The proposed method is trained and tested using two different nuclear cardiac image modalities: $^{13}\text{N}$ -ammonia positron emission tomography (PET) and $^{99m}$ Tc-sestamibi single-photon emission computed tomography (SPECT). MS-ST-UNet attains dice similarity coefficient (DSC) scores of 91.48% and 94.81% for PET LV myocardium (LV-MY) and SPECT LV-MY, respectively. Additionally, the mean-square error (MSE) between predicted rigid registration parameters and ground truth decreases to below $1.4 \times 10^{-2}$ . The experimental findings indicate that the MS-ST-UNet yields notably reduced registration errors and more precise boundary detection for the LV structure compared to existing methods. This joint learning framework promotes mutual enhancement between reorientation and segmentation tasks, leading to cutting edge performance and an efficient image processing workflow.