M2FNet: multi-modality multi-level fusion network for segmentation of acute and sub-acute ischemic stroke

Abstract

Ischemic stroke, a leading cause of death and disability, necessitates accurate detection and automatic segmentation of lesions. While diffusion weight imaging is crucial, its single modality limits the detection of subtle lesions and artifacts. To address this, we propose a multi-modality, multi-level fusion network (M²FNet) that aggregates salient features from different modalities across various levels. Our method uses a multi-modal independent encoder to extract modality-specific features from images of different modalities, thereby preserving key details and ensuring rich features. In order to suppress noise while ensuring the best preservation of modality-specific information, we effectively integrate features of different modalities through a cross-modal encoder fusion module. In addition, a cross-modal decoder fusion module and a multi-modality joint loss are designed to further improve the fusion of high-level and low-level features in the up-sampling stage, dynamically utilizing complementary information from multiple modalities to improve segmentation accuracy. To verify the effectiveness of our proposed method, M²FNet was validated on two public magnetic resonance imaging ischemic stroke lesion segmentation benchmark datasets. Whether single or multi-modality, M²FNet performed better than ten other baseline methods. This highlights the effectiveness of M²FNet in multi-modality segmentation of ischemic stroke lesions, making it a promising and powerful quantitative analysis tool for rapid and accurate diagnostic support. The codes of M²FNet are available at https://github.com/ShannanChen/MMFNet.