Research on UAV Online Visual Tracking Algorithm based on YOLOv5 and FlowNet2 for Apple Yield Inspection*

Accurate monitoring of fruit quantity in apple orchards will allow growers to manage their orchards more efficiently, leading to higher yields. In addition, the rapid and accurate inspection of fruit in the orchard is also one of the basic technologies needed for smart agriculture. This paper proposes a real-time method for apple tracking and yield estimation. UAV carrying RGB camera is used as an inspection platform, which analyzes the video in real time during the inspection. The algorithm is built according to the Tracking-by-Detecting framework, where YOLOV5 target detection model is used to obtain apples’ exact position. Meanwhile, the apples’ position in next frame is estimated according to the optical flow calculated from FlowNet2 model. Then, the predicted position and detected position is matched by the Hungarian algorithm. A dataset close to the actual situation in the orchard is constructed to verify the effectiveness of the proposed method. To improve the detection accuracy of apple under actual scene, the data enhancement strategy of random occlusion and mosaic enhancement is used for model training. As a result, the accuracy of apple detection achieved by the algorithm in this paper is 85.5%, which is 11% higher than previous studies. Besides, it can keep well tracking of detected apples even under the influence of complex occlusion or other factors, and achieve an accuracy of 90.39% in apple yield estimation. More importantly, this algorithm can reach a speed of 20FPS on the experimental platform, which meets the real-time requirements of UAV inspection.