Advanced canopy size estimation in strawberry production: a machine learning approach using YOLOv11 and SAM

This study presents a novel approach for estimating strawberry canopy size by integrating the Segment Anything Model (SAM) with YOLOv11 detection, enhancing accuracy and efficiency in precision agriculture. Traditional methods of canopy size estimation are labor-intensive and frequently inaccurate, posing considerable limitations in agricultural applications. To overcome these issues, our research introduces an innovative integration of SAM’s zero-shot segmentation capabilities with YOLOv11′s advanced detection accuracy, underpinned by a novel prompt selection algorithm. This algorithm automates prompt optimization by using precise detection outputs from YOLOv11 to guide SAM, eliminating the need for extensively annotated datasets required by conventional and supervised segmentation methods. The prompt selection algorithm is proposed in two innovative variants: vanilla and refined. The vanilla approach employs bounding box detections from YOLOv11 plant detection alongside strategically chosen point prompts from fruit detection outputs to enhance segmentation specificity. The refined approach further advances this concept by introducing a hollow concentric structure algorithm to selectively choose background points from regions overlapping fruit detections and preliminary SAM masks. This refinement reduces segmentation errors by identifying non-canopy points, thus improving segmentation reliability. Experimental validation demonstrated that the vanilla approach achieved an Intersection over Union (IoU) of 0.913, while the refined approach reached an even higher IoU of 0.924. Additionally, we integrated Depth Anything v2 (DAv2) to transition from 2D segmentation to robust 3D canopy volume estimation. This comprehensive framework not only improves upon existing segmentation methods but also provides a practical, scalable solution for precision agriculture, showcasing significant advancements in automated canopy analysis.