SAM2-FNet: Medical Image Lesion Segmentation Model Based on Frequency Domain Expert Fusion Network

Abstract

Recent advances in deep learning have improved medical image lesion segmentation. However, existing approaches remain impaired by the challenge of integrating local detail with global semantic context, which often leads to inaccurate boundaries and limited generalization. We propose SAM2-FNet, a frequency-domain expert fusion network addressing these limitations through three innovative components: (1) The frequency-enhanced ensemble module (FEEM) implements spectral decomposition via 2D Fourier transforms, segregating features into high/low-frequency components. These components undergo specialized processing through parallel branches with differential channel attention mechanisms, followed by adaptive fusion to optimize complementary information integration. (2) The fusion expert module (FEM) employs five lightweight subnetworks configured in a multi-expert architecture. During inference, dynamic weighting of expert outputs enables flexible adaptation to lesion heterogeneity, enhancing robustness across varied pathological presentations. (3) A lightweight frequency–spatial integrator (LFSI) is introduced as a substitutable replacement for standard 3 × 3 convolutions within FEEM. It employs a parameterized selective spatial projector for local receptive field modeling, amplifying salient structural responses and suppressing redundancies via a learnable selection mechanism. (4) The local refine module (LRM) bridges encoder-decoder semantic discrepancies through an atrous convolution pyramid structure, effectively recovering fine structural details during boundary reconstruction. (5) Comprehensive evaluations demonstrate SAM2-FNet's superior performance across standard benchmarks. On ISIC2017, the model achieves absolute improvements of 2.6% in DSC and 3.2% in IoU over baseline implementations. For Kvasir-SEG, performance gains reach 3.8% DSC and 4.8% IoU. On the CVC-ClinicDB dataset, the DSC and IoU care increased by 2.8% and 3.9% respectively. Comparative analysis with state-of-the-art approaches (including U-Mambabot and SwinUNETR) reveals consistent advantages in both DSC and IoU metrics, particularly for complex lesion morphologies. These findings suggest that SAM2-FNet offers a novel and effective solution for medical image lesion segmentation, which has high theoretical value and practical application prospects. The source code is available at https://github.com/niubihonghong12345/SAM2-FNET.