Harvesting Point Localization of Tomatoes in a Greenhouse Environment

Abstract

The precise localization of tomatoes is crucial for the stable operation of harvesting robots. To improve the abilities of tomato detection and positioning in a greenhouse, this study proposed a method of harvesting point localization using deep learning. The method focused on tomatoes using a vertical vine training system in a greenhouse. By incorporating the Transformer-based cross stage partial transformer (C3TR) module to capture global dependencies, the proposed method improved the segmentation performance of the you only look once (YOLO)v5s-seg baseline. The harvesting point was obtained by extracting centroid of the segmentation contour. By incorporating fruit shape priors, this study proposes a tomato completion method under occlusion conditions. The experimental results show the improved model achieves 90.7%, 90.6%, 96.4%, and 0.907 in terms of precision, recall, mean average precision (mAP)@50, and F1 score, respectively. The performance of the improved model is superior to that of the YOLO series baseline models. This study designed the 2-D, 3-D localization, and robotic arm grasping experiment. The average pixel error of the 2-D experiment is 2.531, and the average space error is about 1.0 cm, which verifies the effectiveness and practicality of the method. This study can provide a theoretical basis and technical support for the development of intelligent tomato harvesting equipment.