Orange yield estimation using object tracking and 3D reconstruction

Abstract

The labor-intensive nature of agriculture, particularly in tasks such as yield estimation of fruits, is a significant challenge. Yield estimation is crucial for the better management of the resources and for taking adequate measures for the transportation, storage, and export of the fruits. It also helps the farmers to estimate the total pricing of the yield. However, counting fruits directly on trees for yield estimation presents an obstacle due to their dispersed nature and often dense foliage. Therefore, we propose that reasonably accurate fruit yield estimation can be automated with a handheld camera. The dataset is curated by capturing and annotating 1451 images of orange trees. The dataset is augmented and processed in different ways to evaluate the performance of YOLOv8 for the detection of oranges. Then the Byte tracker is deployed to track oranges in consecutive video frames. Further, we have classified the fruits into two categories, ripe and unripe using MobileVit. The 2D fruits detected by YOLOv8 are projected to a 3D space for a more detailed analysis of the scene. Subsequently, the clustering algorithm is applied to the 3D projections of the detected objects to estimate per tree yield. On images, YOLOv8 nano has achieved a precision of 78.2% and recall of 69.7% on the test set. Moreover, for ripeness stage classification, MobileVit has achieved an accuracy of 97.8% and 86.7% on a test set containing 2 classes and 3 classes, respectively. Testing our proposed solution on videos shows that the algorithm is achieving good results on trees with less leaf occlusion. This paper demonstrates that preprocessing techniques can aid the detection model to achieve high detection rates. Furthermore, per tree yield of an orange orchard can be estimated by using video input. This offers an automated solution to the laborious task of fruit yield estimation in agricultural settings, that can help to optimize orange production.