Hierarchical multi-scale Mamba generative adversarial network for multi-modal medical image synthesis

Abstract

In recent years, the rapid advancement of medical imaging technology has placed higher demands on diagnostic accuracy, making multi-modal medical image synthesis an essential pathway for comprehensive diagnosis. Although Generative Adversarial Networks (GANs) have achieved some progress in the field of medical image synthesis, their reliance on single-modality mapping often hampers the effective capture of complex contextual relationships between modalities, thereby limiting the quality and precision of the synthesized images. We propose a novel multi-modal medical image synthesis model, HMS-MambaGAN, which adopts a hierarchical multi-scale structure that effectively captures and fuses both global and local features through the innovative design of channel ConvNext Mamba (ConvMamba) blocks, Episodic Bottleneck, HMSModule, and a dual-decoder structure. Additionally, we have designed a loss function based on the Gray-Level Gradient Co-occurrence Matrix (GLGCM), incorporating gradient information to enhance the texture and structural details of the synthesized images, while a diffusion model is utilized as an auxiliary component to synthesize additional target domain images, further improving overall image quality and detail representation. Our results demonstrate that HMS-MambaGAN significantly outperforms current state-of-the-art models on multi-contrast MRI, MRI-CT, and CT-PET datasets. Our code is publicly available at https://github.com/jlw9999/HMS-MambaGAN.