Anatomy-Guided, Modality-Agnostic Segmentation of Neuroimaging Abnormalities

Abstract

Magnetic resonance imaging (MRI) offers multiple sequences that provide complementary views of brain anatomy and pathology. However, real-world datasets often exhibit variability in sequence availability due to clinical and logistical constraints. This variability complicates radiological interpretation and limits the generalizability of machine learning models that depend on a consistent multimodal input. Here, we propose an anatomy-guided, modality-agnostic framework to assess disease-related abnormalities in brain MRI, leveraging structural context to ensure robustness in diverse input configurations. Central to our approach is Region ModalMix (RMM), an augmentation strategy that integrates anatomical priors during training to improve model performance under missing or variable modality conditions. Using the BraTS 2020 dataset (n = 369), our framework outperformed state-of-the-art methods, achieving a 9.68 mm average reduction in 95th percentile Hausdorff Distance (HD95) and a 1.36 percentage point improvement in Dice Similarity Coefficient (DSC) over baselines with only one available modality. To evaluate out-of-distribution generalization, we tested RMM on the MU-Glioma-Post dataset (n = 593), which includes heterogeneous post-operative glioma cases. Despite distribution shifts, RMM maintained strong performance, reducing HD95 by 18.24 mm and improving DSC by 9.54% points in the most severe missing-modality scenario. Our framework is applicable to multimodal neuroimaging pipelines, enabling more generalizable abnormality detection under heterogeneous data availability.