Enhanced visual detection of litchi fruit in complex natural environments based on unmanned aerial vehicle (UAV) remote sensing

Abstract

Rapid and accurate detection of fruits is crucial for estimating yields and making scientific decisions in litchi orchards. However, litchis grow in complex natural environments, characterized by variable lighting, severe occlusion from branches and leaves, small fruit sizes, and dense overlapping, all of which pose significant challenges for accurate detection. This paper addressed this problem by proposing a method that combines unmanned aerial vehicle (UAV) remote sensing and deep learning for litchi detection. A remote sensing image dataset comprising litchi fruit was first constructed. Subsequently, an improved algorithm, YOLOv7-MSRSF, was developed. Experimental results demonstrated that YOLOv7-MSRSF’s mean average precision (mAP) reached 96.1%, outperforming YOLOv7 and pure transformer algorithms by 3.7% and 20.6%, respectively. Tests on randomly selected 24 images demonstrated that integrating the Swin-transformer module into YOLOv7 improved litchi fruit detection accuracy under severe occlusion, dense overlapping, and variable lighting by 19.55%, 6.63%, and 13.94%, respectively. YOLOv7-MSRSF showed further improvements in these three complex conditions, with detection accuracy increasing by 26.99%, 9.82%, and 18.68%, respectively, reaching 89.16%, 97.79%, and 95.56%. Furthermore, the Real-ESRGAN algorithm significantly enhanced the YOLOv7-MSRSF model’s recognition accuracy of objects in low-resolution images captured by high-altitude drones. The average detected accuracy of three images collected at 27.5 m above the canopy reached a high value of 82.2%, which was improved by 70.6% compared with that (11.6%) before super-resolution processing. The proposed method offered valuable guidance for detecting small, dense agricultural objects in large-scale, complex natural environments.