Vessel-SAM2: Adapting Segment Anything 2 for Patch-Free Retinal Vessel Segmentation in Ultra-High Resolution Fundus Images

Abstract

Accurate automatic segmentation of blood vessels in ophthalmic images is crucial for the early diagnosis of many diseases. These images are typically high-resolution and contain intricate details of fine terminal vessels. However, most existing deep learning methods operate on lower resolutions, which limits their segmentation accuracy. Learning directly from high-resolution images faces significant challenges, as the computational overhead required by existing complex segmentation decoders can be impractical. To address these challenges, we propose Vessel-SAM2, a retinal vessel segmentation network based on Segment Anything 2 (SAM2), capable of performing end-to-end segmentation at an ultra-high resolution of 2048 × 2048 without the need for cumbersome patching. Vessel-SAM2 fine-tunes the pretrained Hiera of SAM2 using adapters in a parameter-efficient manner, while its decoder incorporates an efficient attention aggregation mechanism. Extensive experiments demonstrate the superior performance of Vessel-SAM2.