2MSPK-Net: A nuclei segmentation network based on multi-scale, multi-dimensional attention, and SAM prior knowledge

Refined nuclei segmentation is of great significance for diagnosing the pathological conditions of tumor tissues. Although existing encoder–decoder networks have achieved remarkable progress in nuclei segmentation tasks, practical applications still encounter obstacles, especially for challenging issues such as highly dense nuclei targets and the ambiguity of boundaries between inter-class features, resulting in unsatisfactory segmentation accuracy. In this work, a novel encoder–decoder architecture was proposed to address these issues. Specifically, we first proposed a multi-scale and multi-dimension attention module to capture the contextual dependencies between individual pixels and the overall pixels, where in cross-scale learning was achieved by fusing different scale feature information of the encoding layer. Secondly, we integrated the prior knowledge of SAM into nuclei images to enhance the network’s ability to distinguish fuzzy features. To the best of our knowledge, this was the first attempt to utilize the prior knowledge of SAM to optimize nuclei segmentation tasks. Furthermore, the network was guided to supplement missing detailed features through a reverse erasing strategy and cross-layer information flow. Comprehensive experiments illustrated that the proposed method achieved MIoU improvements of 1.26% and 0.94% on the MoNuSeg and TNBC datasets, respectively, over several SOTA methods, indicating its great potential as a backbone for cancer nuclei segmentation. Code: https://github.com/ThirteenYue/2MSPK-Net.