ScribSAM: A robust scribble-supervised framework for spatiotemporal segmentation of breast lesions in ultrasound videos

Abstract

Automated segmentation of breast lesions in ultrasound videos is critical for clinical applications but remains hindered by the reliance on expensive pixel-wise annotations. While scribble supervision offers a user-friendly alternative, its potential for ultrasound video segmentation remains underexplored. In this regard, we propose ScribSAM, a novel scribble-supervised framework built on the Medical Segment Anything Model (MedSAM) for robust ultrasound video segmentation. ScribSAM integrates two key innovations: a flow-guided scribble propagation module that leverages optical flow to efficiently propagate sparse scribble annotations across frames while preserving temporal consistency, and a bidirectional cross-attention module that fuses MedSAM’s global ViT embeddings with 3D CNN local-temporal embeddings for enhanced spatiotemporal feature learning. Extensive experiments on the scribble-annotated variants of two ultrasound video datasets, BUV2022 and US-VOS, demonstrate ScribSAM’s superiority. It surpasses state-of-the-art scribble-supervised methods by 5.53% (BUV2022) and 8.31% (US-VOS) in Dice score, outperforming some fully supervised methods and substantially narrowing the gap with the best fully supervised methods, while using only 4% of the annotated pixels. Code and dataset will be released at https://github.com/003-GH/ScribSAM.