MACA-Net: Multi-aperture curvature aware network for instance-nuclei segmentation

Nuclei instance segmentation is one of the most challenging tasks and is considered the first step in automated pathology. The challenges stem from technical biological variations, and high cellular density that lead adjacent nuclei to form perceptual boundaries. This paper demonstrates that a multi-aperture representation encoded by the fusion of Swin Transformers and Convolutional blocks improves nuclei segmentation. The loss function is augmented with the curvature and centroid consistency terms between the growth truth and the prediction to preserve morphometric fidelity and localization. These terms are used to panelize for the loss of shape localization (e.g., a mid-level attribute) and mismatches in low and high-frequency boundary events (e.g., a low-level attribute). The proposed model is evaluated on three publicly available datasets: PanNuke, MoNuSeg, and CPM17, reporting improved Dice and binary Panoptic Quality (PQ) scores. For example, the PQ scores for PanNuke, MoNuSeg, and CPM17 are 0.6888 ± 0.032, 0.634 ± 0.003, and 0.716 ± 0.002, respectively. The code is located at https://github.com/Siyavashshabani/MACA-Net.