Fine-Tuning SAM for Forward-Looking Sonar With Collaborative Prompts and Embedding

Abstract

The segment anything model (SAM) represents a significant advancement in semantic segmentation, particularly for natural images, but encounters notable limitations when applied to forward-looking sonar (FLS) images. The primary challenges lie in the inherent boundary ambiguity of FLS images, which complicates the use of prompt strategies for accurate boundary delineation, and the lack of effective interaction between prompts and image features. In this letter, we introduce a collaborative prompting (CP) strategy to address these issues by generating dense prompt embeddings and sonar tokens that focus on contour and boundary features, thereby replacing the original dense prompt embedding and intersection over union (IoU) token. To further enhance segmentation, we use embedding compensation techniques based on Mamba and Kolmogorov–Arnold network (KAN), which increase boundary information to image embeddings and improve the fusion of prompts within image embeddings. We conducted comprehensive experiments, including comparative analyses and ablation studies, to validate the superiority of our proposed approach. Results show that our method significantly improves segmentation performance for FLS images, effectively addressing boundary ambiguity and optimizing prompt utilization. The source code and dataset will be available on https://github.com/darkseid-arch/FLSSAM