Real-time detection and localization using SSD method for oyster mushroom picking robot*

Mushroom picking robot could significantly decrease the labor cost during production process. As well as humans, robots need to know what the mushroom is, where it is, and finally harvest it. This paper proposes an accurate and real-time capable object detection and localization approach for the use on oyster mushroom picking robot. Detection information of a neural network is combined with depth information from a RGB-D camera, which is mounted on the picking robot platform. The SSD object detection algorithm is used as convolutional neural network. In order to find the detected object in the 3D environment, the depth image based on binocular and structured light principles is also used to locate the precise position. In the evaluation part, the labelled images (4000 training, 300 validation and 300 test) of this study are tested. The results of picking robot experiments in the greenhouse showed that the recognition FI score of oyster mushroom had reached 0.951, and the 3D localization error of the vision system was only 2. 43mm, which guarantee the real-time performance and positioning requirements of oyster mushroom picking robots.