A 3D UNet-based fusion network for brain tumor segmentation with missing modalities

Abstract

Multimodal magnetic resonance imaging provides complementary information for brain tumor segmentation, significantly enhancing the accuracy of diagnosis and prognosis. However, the common issue of missing modalities in clinical practice severely undermines the performance of existing methods, as they predominantly rely on complete multimodal data and struggle to effectively handle dynamic inter-modality correlations and tumor region specificity. To address this challenge, we propose a novel fusion network based on 3D U-Net, termed MPDF-UNET. Its core innovation lies in the introduction of the Modality Priors and Dynamic Features fusion (MPDF) module, which adaptively learns the unique representations of different MRI modalities under conditions of partial modality loss while effectively integrating complementary information across modalities. Additionally, we develop a modality combination sampling strategy that dynamically adjusts the distribution of modality combinations in the training data. This strategy encourages the network to fully exploit prior knowledge from each modality, thereby enhancing model robustness under conditions of missing modalities. To mitigate the impact of missing modality-associated dynamic feature information, we further propose a feature loss function. By imposing constraints on dynamic features, this loss function facilitates the learning of modality priors, alleviating the degradation of the network’s representational capacity caused by missing modalities. Experiments conducted on BRATS2018 and BRATS2020 benchmark datasets demonstrate the superiority of MPDF-UNET. Notably, the model achieves significant improvements in the fine-grained segmentation of enhancing tumors, surpassing current SOTA. Specifically, on BRATS2018 dataset, our method improves the Dice score of enhancing tumor segmentation by 7.78 % on average compared to the best-performing baseline Region-aware Fusion Network (RFNet), demonstrating superior robustness under missing modalities. This work provides a reliable solution for incomplete or resource-limited multimodal data in clinical settings, demonstrating significant practical value.