MLDCGAN: A multimodal latent diffusion conditioned GAN model for accelerated and high-fidelity MRI-CT synthesis in radiotherapy planning

Abstract

Magnetic resonance imaging (MRI) offers significant advantages in soft tissue contrast. However, it cannot directly provide electron density information for radiotherapy, relying instead on time-consuming and error-prone MRI-CT image registration. Synthetic CT (sCT) technology, which directly generates CT images from MRI, is pivotal for achieving only MRI-based radiotherapy. However, existing synthesis methods based on generative adversarial network (GAN) and diffusion models face challenges such as prolonged inference times and insufficient utilization of multimodal information, which severely hinder the clinical application of synthetic images. In this study, we propose a novel Multimodal Latent Diffusion Conditioned GAN (MLDCGAN) Model. First, we design a non-parametric non-Gaussian complex denoising distribution based on a conditional GAN, employing a multimodal distribution to achieve large-step denoising. This is combined with a pre-trained autoencoder to compress the image into a low-dimensional latent space, significantly reducing inference time. Second, we fully leverage multimodal MRI information by constructing a local refinement conditional generator with multimodal inputs, including T1-Weighted (T1W), T2-Weighted (T2W), and Mask images. The generator is enhanced by an adaptive weighted multi-sequence fusion module and an enhanced cross-attention module, significantly improving the structural consistency and detail fidelity of the generated sCT images. Finally, by jointly optimizing the style loss and content loss, we ensure the perceptual quality and clinical accuracy of the synthetic images. Experimental results demonstrate that MLDCGAN outperforms existing state-of-the-art methods on both public and private datasets, showing significant improvements in both image quality and inference speed. Subjective evaluations from multiple experienced clinicians indicate that the generated sCT images exhibit no significant difference from real CT in terms of key anatomical structure clarity and overall quality (P > 0.05). Further assessments of clinical target delineation and dose distribution confirm that sCT retains anatomical features well and provides dose distributions consistent with real CT, ensuring the reliability of dose calculations in radiotherapy planning. This study provides a more reliable and efficient technical foundation for achieving only MRI-based radiotherapy. It is expected to assist clinicians in developing more precise radiotherapy plans, ultimately improving treatment outcomes in future clinical practice.