RESAMPL-UDA: Leveraging foundation models for unsupervised domain adaptation in biomedical images

Abstract

Large annotated datasets and new models have led to significant improvements in supervised semantic segmentation. On the other side, Unsupervised Domain Adaptation (UDA) for Semantic Segmentation is still an arduous open research topic. While new ideas frequently come out based on recent findings, best methods still rely on basic techniques such as the use of pseudo-labels on target for self-training. Nonetheless, such methods fail when applied to difficult UDA cases like Biomedical Images where the domain shift is too high, leading to pseudo-labels of poor quality. In this work, we propose RESAMPL-UDA (REfined SAM-based Pseudo Label UDA), an unsupervised domain adaptation method that effectively integrates zero-shot predictions from the Segment Anything (SAM) model. Given the high complexity and variability of biomedical images, SAM alone often produces detailed segmentations without necessarily capturing the intended structures. To address this, our method involves training a dedicated refinement network on source domain data to selectively enhance SAM-generated masks. These refined segmentations then serve as reliable pseudo-labels within our UDA framework, significantly facilitating the adaptation process. Experiments on 8 adaptation cases demonstrate that our method outperforms the state of the art. In addition, we extend successfully our work to Source-Free Unsupervised Domain Adaptation, demonstrating its versatility. The code is available : https://github.com/alex-stenger/RESAMPL-UDA.