Segment Any Tissue: One-shot reference guided training-free automatic point prompting for medical image segmentation

Abstract

Medical image segmentation frequently encounters high annotation costs and challenges in task adaptation. While visual foundation models have shown promise in natural image segmentation, automatically generating high-quality prompts for class-agnostic segmentation of medical images remains a significant practical challenge. To address these challenges, we present Segment Any Tissue (SAT), an innovative, training-free framework designed to automatically prompt the class-agnostic visual foundation model for the segmentation of medical images with only a one-shot reference. SAT leverages the robust feature-matching capabilities of a pretrained foundation model to construct distance metrics in the feature space. By integrating these with distance metrics in the physical space, SAT establishes a dual-space cyclic prompt engineering approach for automatic prompt generation, optimization, and evaluation. Subsequently, SAT utilizes a class-agnostic foundation segmentation model with the generated prompt scheme to obtain segmentation results. Additionally, we extend the one-shot framework by incorporating multiple reference images to construct an ensemble SAT, further enhancing segmentation performance. SAT has been validated on six public and private medical segmentation tasks, capturing both macroscopic and microscopic perspectives across multiple dimensions. In the ablation experiments, automatic prompt selection enabled SAT to effectively handle tissues of various sizes, while also validating the effectiveness of each component. The comparative experiments show that SAT is comparable to, or even exceeds, some fully supervised methods. It also demonstrates superior performance compared to existing one-shot methods. In summary, SAT requires only a single pixel-level annotated reference image to perform tissue segmentation across various medical images in a training-free manner. This not only significantly reduces the annotation costs of applying foundational models to the medical field but also enhances task transferability, providing a foundation for the clinical application of intelligent medicine. Our source code is available at https://github.com/SnowRain510/Segment-Any-Tissue.