Medical volume CT-to-MRI translation with multi-dimensional diffusion architecture

Abstract

In recent years, there has been a proliferation of novel techniques utilizing neural networks for the generation of images. These models are called generative networks. The diffusion model represents the most popular generative network. It outperforms others in numerous domains, such as image super-resolution, image in-painting, and generating images based on textual descriptions, among others. However, most papers research two-dimensional (2D) image generation. Few focus on three-dimensional (3D) aspects such as video and volumetric data generation. The objective of our research is to develop a method for translating Computed Tomography Volumes (CT Volumes) into Magnetic Resonance Imaging Volumes (MRI Volumes). To achieve this goal, it is necessary to address four challenges: large amounts of memory required, long inference time, short data amounts, and the inaccuracy of the resulting details. Consequently, we use a 3D latent diffusion model and a 2D diffusion model to overcome these challenges. Furthermore, unlike traditional padding methods that pad input first, we introduce a module, defined as a scalable module, which allows the input to adapt different shapes in each layer of the model. We compare our model with the state-of-the-art methods. The experimental results demonstrate that our method outperforms those methods.