DGSSA: Domain generalization with structural and stylistic augmentation for retinal vessel segmentation

Abstract

Retinal vascular morphology plays a crucial role in diagnosing diseases such as diabetes, glaucoma, and hypertension, making accurate segmentation of retinal vessels essential for early intervention. Traditional segmentation methods assume that training and testing data share similar distributions, which can lead to poor performance on unseen domains due to domain shifts caused by variations in imaging devices and patient demographics. This paper presents a novel approach, DGSSA, for retinal vessel image segmentation that enhances model generalization by combining structural and stylistic augmentation strategies. We utilize a space colonization algorithm to generate diverse vascular-like structures that closely mimic actual retinal vessels, which are then used to generate pseudo-retinal images with an improved Pix2Pix model, allowing the segmentation model to learn a broader range of structure distributions. Additionally, we utilize PixMix to apply random photometric augmentations and introduce uncertainty perturbations, enriching the stylistic diversity of fundus images and further improving the model’s robustness and generalization across varying imaging conditions. Our framework, which employs a DeepLabv3+ model with a MobileNetV2 backbone as its segmentation network, has been rigorously evaluated on four challenging datasets—DRIVE, CHASEDB1, HRF, and STARE—achieving Dice Similarity Coefficient (DSC) of 78.45%, 78.62%, 72.66% and 82.17%, respectively, with an average DSC of 77.98%. These results demonstrate that our method surpasses existing approaches, validating its effectiveness and highlighting its potential for clinical application in automated retinal vessel analysis.