Detection and picking point localization of grape bunches and stems based on oriented bounding box

Abstract

Accurate positioning of the picking point is the key to achieving automated grape picking. Grape recognition and picking point positioning in natural environments often use conventional recognition or segmentation methods. These methods require long model training and poor positioning effects and are unsuitable for mobile device deployment. This study proposes an improved YOLO11-GS (YOLO11-Grape and Stem) model based on YOLO11n-OBB. First, dynamic convolution and Ghost module are fused to replace Bottleneck in C3k2 module. BiFPN is used as a new feature fusion method, combined with parameter-free attention SimAM, to improve feature extraction capability while reducing model parameters. Secondly, oriented bounding boxes are introduced into grape stem recognition to better fit the shape features of stems, and oriented bounding boxes and geometric methods are combined to quickly and accurately locate grape picking points and picking postures. Finally, ByteTrack and BoT-SORT tracking algorithms are used for tracking grape bunches and stems. Experimental results show that the YOLO11-GS algorithm achieves 87.7% and 89.5% precision and mAP in grape recognition, which are 3.4% and 2.0% higher than the standard YOLO11n-OBB, 8.1% and 11.7% higher than YOLO11n without oriented bounding box, and 28.3% lower Params. The proposed picking point positioning and pose estimation method achieves an average error of 8.63 pixels and 5.08°, and the average tracking error of the picking point is 8.758 pixels. The deployment of the YOLO11-GS model on mobile devices shows that it can accurately and efficiently complete the identification of grape fruits and stems and the positioning of picking points, providing necessary support for the development of automated grape picking technology.