Attention-based unsupervised prompt learning for SAM in leaf disease segmentation

Abstract

In modern agriculture, leaf disease segmentation is crucial for crop disease management and yield improvement. Since most deep learning-based segmentation models require extensive annotations, pursuing unsupervised methods becomes a practical solution. Although advanced models like the Segment Anything Model (SAM) can produce precise class-agnostic masks without annotations, their reliance on human-interactive prompts limits their utility to unsupervised disease segmentation tasks. To address this limitation, we present UPLS, a novel three-stage framework that leverages unsupervised prompt learning to enable SAM for leaf disease segmentation. This method employs unsupervised contrastive learning to progressively segment leaf and disease areas and automatically generate disease prompts for SAM. Concretely, for fine and intricate lesions, we utilize an improved high-frequency attention mechanism to extract high-frequency features from the leaf area, and construct contrastive losses between global priors and foreground/background features. Furthermore, we devise a strategy to automatically generate disease-related prompts from the segmented leaf and initial lesion regions, enabling SAM to refine disease boundaries without human intervention. Experiments on three public plant disease segmentation datasets show that UPLS outperforms existing non-fully supervised segmentation methods in accuracy and robustness. Source code is available at https://github.com/Tianluda/UPLS.