A conflict-free multi-modal fusion network with spatial reinforcement transformers for brain tumor segmentation.

Abstract

Brain gliomas are a leading cause of cancer mortality worldwide. Existing glioma segmentation approaches using multi-modal inputs often rely on a simplistic approach of stacking images from all modalities, disregarding modality-specific features that could optimize diagnostic outcomes. This paper introduces STE-Net, a spatial reinforcement hybrid Transformer-based tri-branch multi-modal evidential fusion network designed for conflict-free brain tumor segmentation. STE-Net features two independent encoder-decoder branches that process distinct modality sets, along with an additional branch that integrates features through a cross-modal channel-wise fusion (CMCF) module. The encoder employs a spatial reinforcement hybrid Transformer (SRHT), which combines a Swin Transformer block and a modified convolution block to capture richer spatial information. At the output level, a conflict-free evidential fusion mechanism (CEFM) is developed, leveraging the Dempster-Shafer (D-S) evidence theory and a conflict-solving strategy within a complex network framework. This mechanism ensures balanced reliability among the three output heads and mitigates potential conflicts. Each output is treated as a node in the complex network, and its importance is reassessed through the computation of direct and indirect weights to prevent potential mutual conflicts. We evaluate STE-Net on three public datasets: BraTS2018, BraTS2019, and BraTS2021. Both qualitative and quantitative results demonstrate that STE-Net outperforms several state-of-the-art methods. Statistical analysis further confirms the strong correlation between predicted tumors and ground truth. The code for this project is available at https://github.com/whotwin/STE-Net.