Robust keypoint-based method for peduncle pose estimation in unstructured environments

Abstract

Visual detection for automated fruit harvesting in unstructured environments constitutes a critical technical challenge, especially for fruit peduncles, which exhibit greater sensitivity to environmental factors than the fruits themselves. To address this challenge, this paper proposes a top-down keypoint detection method for pepper peduncles in unstructured environments. The proposed method enables accurate estimation of peduncle poses. The first step of the research involves validating different object detection models and employing ones to identify the bounding boxes of pepper peduncles. Subsequently, a new keypoint detection model based on the Lite Vision Transformer is proposed, leveraging the Transformer’s capacity to capture long-range spatial and semantic dependencies. Experimental results on the pepper dataset collected in unstructured environments demonstrate that the proposed model achieves an AP⁵⁰ of 94.6 %. This performance surpasses multiple state-of-the-art keypoint detection methods while maintaining lightweight parameters and low computational complexity. Moreover, a series of tests reveals that the proposed method outperforms other algorithms in complex environments, especially in occlusion scenarios. Finally, a comprehensive evaluation of the top-down approach is conducted, examining the influence of object detection and keypoint detection models on overall performance. The proposed keypoint detection model achieves the highest performance, with a detection speed of 9.38 FPS when using YOLOv8s as the object detection model, and an AP⁵⁰ of 83.6 % when using YOLOv8l. Experiments conducted in real unstructured environments demonstrated the robustness of the proposed method, effectively detecting the posture of dense and occluded chili pepper peduncles. This research can be extended to the detection of fruit peduncles in other crops, providing a foundation for pose estimation of fruit peduncles in complex environments.