MSPCNF-Net: Multi-scale parallel cross-neighborhood fusion network for medical image segmentation

Abstract

Transformer-based architectures have emerged to deal with inherent limitations of CNNs in catching long-range dependencies for image analysis tasks. However, these approaches generally struggle to process both global and local context information simultaneously. Therefore, the paper establishes a novel dual encoder-decoder framework termed Multi-Scale Parallel Cross-Neighborhood Fusion Network (MSPCNF-Net). It develops a dual-branch network to leverage CNN and Transformer components for acquiring local and global features at multiple scales. For optimizing this feature fusion from these dual-branch encoders, two specialized modules are designed, including the Bidirectional Window Perception Attention (BWPA) module and the Bidirectional Cross Attention (BCA) module. In addition, a Neighborhood Spatial Attention (NSA) module incorporating Gumbel-softmax is implemented by proximal pixels, which facilitates the processing of fine-grained local information and emphasizes key features with lower computational demands. Experiments are performed on four datasets with three distinct tasks including abdominal organ, cardiac organ, and retinal vessel segmentation, which indicate that MSPCNF-Net attains superior effectiveness compared to current well-known methods.