Research on multi-layer model attitude recognition and picking strategy of small tomato picking robot

In recent years, the prospect of robotic picking in greenhouse tomato production has attracted widespread attention, but the research on its replacement for manual picking is still insufficient. Compared with manual picking, there is still a large gap between robots regarding precision and efficiency. In this study, the success rate and efficiency of small tomato robotic picking are significantly improved by designing a multilayer model of attitude recognition and localization method and picking strategy. First, based on the efficient picking robot, an advanced detection model and picking prioritization planning strategy were developed to determine the optimal picking order based on the relationship between small tomato bunches and ripe fruits. Subsequently, the location of the shear point is roughly predicted based on the detected attitude information. A step-by-step approximation strategy approaches the target point, and then the optimized Sm-ICNet model is used to segment the fruit stalks accurately. Combined with the connected region calculation, region masking and depth image filtering algorithms, the target is accurately localized to reduce the interference of branches and leaves. Finally, a highly fault-tolerant end-effector system focusing on shearing fruit peduncles is designed, including a 3D point cloud fusion rigid-body matrix shearing positional method and corresponding end-effector, which ensures the stable shearing of small tomato bunches with different postures. The results of agricultural greenhouse experiments show that the number of parameters based on the Sm-YOLOv7-Tiny model is only 5.756MB with 86.81% mAP, and the mIoU of the Sm-ICNet model reaches 93.7%, which meets the requirements of real-time operation. The optimized harvesting strategy significantly reduces the fruit damage rate, increases the harvesting success rate to 82.1%, and shortens the average harvesting time of a single bunch to 9.8 s. The results of this study show that the application of robotic systems in the tomato industry will gradually become a reality as harvesting technology continues to advance.