Cauliflower centre detection and 3-dimensional tracking for robotic intrarow weeding

Abstract

Mechanical weeding is an important part of integrated weed management. It destroys weeds between (interrow) and in (intrarow) crop rows. Preventing crop damage requires precise detection and tracking of the plants. In this work, a detection and tracking algorithm was developed and integrated on an intrarow hoeing prototype. The algorithm was developed and validated on 12 rows of 950 cauliflower plants. Therefore, a methodology was provided to automatically generate a label based on the crop plants’ Global Navigation Satellite System (GNSS) position during data collection with a robot platform. A CenterNet architecture was adjusted for plant centre detection by comparing different encoder networks and selecting the optimal hyperparameters. The monocular camera projection error of the plant centre detections in pixel to 3D coordinates was evaluated and used in a position- and velocity-based tracking algorithm to determine the timing for intrarow hoeing knife actuation. A dataset of 53k labelled images was created. The best CenterNet model resulted in an F1 score on the test set of 0.986 for detecting cauliflower centres. The position tracking had an average variation of 1.62 cm. Velocity tracking had a standard deviation of 0.008 \(\mathrm {m\,\,s^{-1}}\) with respect to the robot’s operational target velocity. Overall, the entire integration showed effective actuation of the prototype in field conditions. Only one false positive detection occurred during operation in two test rows of 135 cauliflowers.