Improving picking efficiency under occlusion: Design, development, and field evaluation of an innovative robotic strawberry harvester

Abstract

Robotic harvesting has long been seen as the potential alternative to manual harvesting in the strawberry industry. However, despite much progress made in the harvesting process from detection to picking, these technologies are not yet commercially viable. One of the limiting factors for increased performance is fruit occlusion in canopies, particularly in the open-field environments. There has been only limited studies on active occlusion handling/removal techniques during robotic picking. This paper presents the development and evaluation of a strawberry harvesting robot focusing on occlusion handling in open-field environments using vision-based occlusion information and novel end-effector design. The robot was composed of an integrated machine vision system based on deep learning techniques, a 6 DOF manipulator, and an innovative end-effector equipped with a fan system, and a mobile platform. Based on the classification of detected strawberries (‘not occluded’ or ‘occluded’), the robotic platform followed specific steps for directly picking the strawberries (if not occluded) or removing/dispersing the occlusion over the strawberries (if occluded) and subsequently picking them. The effectiveness of this harvesting robot including fruit recognition & localization, and picking method was evaluated using multiple experiments in both the simulation field and the real field. The results showed that the mean average precision in strawberry detection was 80.5% and classification accuracy was 93.2%. Picking efficiency of the robot was enhanced substantially by the use of fan system. In an outdoor strawberry field, the robot achieved a picking rate of 58.1% without fan system, which increased to 73.9% with the fan system (a 15.8% increase in fruit picking rate). It was found that the average processing time of machine vison system was 6.26 s and the overall average time to pick single strawberry with the fan system for removing occlusion was 20.1s.