OneRef-Seg: A Single-Reference Framework for Efficient Medical Image Segmentation

Abstract

While the Segment Anything Model (SAM) has revolutionized medical image segmentation with its zero-shot capabilities, its reliance on manual, per-image prompting (e.g., points or boxes) imposes a prohibitive interaction burden and creates a profound intent gap between sparse spatial cues and complex anatomical semantics. To address these challenges, we propose OneRef-Seg, a unified task-level single-reference conditioning framework. OneRef-Seg combines the flexibility of reference-guided interaction with single-example task-level conditioning; at deployment, a unified OneRef-Seg model, jointly trained across predefined binary segmentation tasks, receives a single annotated reference image-mask pair that specifies the current anatomical target and automatically segments unseen target images from the same task without per-target spatial prompting. Technically, we introduce a Reference-Target Feature Fusion Module (RTFM) designed to explicitly bridge the semantic gap through dense contextual alignment. The RTFM utilizes a cascaded reasoning mechanism to migrate reference priors into the target feature space, effectively bypassing the fragility of heuristic prompt generation. Extensive evaluations on the REFUGE, ACDC, and Fluoroscopy benchmarks demonstrate that OneRef-Seg achieves performance comparable to single-round prompt-driven SAM variants while significantly reducing user intervention. Notably, OneRef-Seg remains highly competitive against fully supervised specialist models, offering a practical and efficient solution for high-throughput medical AI workflows.