A Novel Perceptual Constrained cycleGAN With Attention Mechanisms for Unsupervised MR-to-CT Synthesis

Abstract

Radiotherapy treatment planning (RTP) requires both magnetic resonance (MR) and computed tomography (CT) modalities. However, conducting separate MR and CT scans for patients leads to misalignment, increased radiation exposure, and higher costs. To address these challenges and mitigate the limitations of supervised synthesis methods, we propose a novel unsupervised perceptual attention image synthesis model based on cycleGAN (PA-cycleGAN). The innovation of PA-cycleGAN lies in its model structure, which incorporates dynamic feature encoding and deep feature extraction to improve the understanding of image structure and contextual information. To ensure the visual authenticity of the synthetic images, we design a hybrid loss function that incorporates perceptual constraints using high-level features extracted by deep neural networks. Our PA-cycleGAN achieves notable results, with an average peak signal-to-noise ratio (PSNR) of 28.06, structural similarity (SSIM) of 0.95, and mean absolute error (MAE) of 46.90 on a pelvic dataset. Additionally, we validate the generalization of our method by conducting experiments on an additional head dataset. These experiments demonstrate that PA-cycleGAN consistently outperforms other state-of-the-art methods in both quantitative metrics and image synthesis quality.