PhenoSR: Enhancing organ-level phenotyping with super-resolution RGB UAV imagery for large-scale field experiments

Abstract

Organ-level phenotyping is critical for crop breeding and precision farming by providing information directly associated with yield and quality. Unmanned aerial vehicles (UAVs) are widely utilized in large-scale field experiments for their versatile image collection capabilities. However, RGB images captured at high altitudes often lack the resolution for accurate organ-level phenotyping, as collection efficiency is prioritized. Deep learning-based image super-resolution (SR) methods can enhance image resolution, but they usually fail to address the challenge of obtaining paired low-resolution (LR) and high-resolution (HR) data for training under field conditions. Moreover, the varying significance of organ-level phenotyping across different regions in UAV images is often neglected, slowing down reconstruction. To overcome these challenges, a degradation model and a multiscale scaling strategy were proposed to generate paired datasets. Then, a semantic score was introduced to identify the significance of image regions for organ-level phenotyping. Finally, an SR algorithm (PhenoSR) based on a coarse-refined architecture was proposed to recover organ textures. PhenoSR recovered wheat organ textures in UAV images collected at flight heights ranging from 10 to 40 m. Compared to LR images, the natural image quality evaluator (NIQE) and Fréchet inception distance (FID) metrics decreased by 71.37 % and 21.53 %, respectively, while improving hyperIQA by 39.36 %. PhenoSR outperformed eight SR algorithms, achieving a 12.31 % reduction in FID and a 25.53 % improvement in hyperIQA on average. Moreover, PhenoSR enhanced organ-level wheat phenotyping tasks, such as plot segmentation, spike counting, flowering spike detection, and awn morphology identification, and can be extended to other crops and multispectral imagery. This study presents an innovative and universal technology for enhancing organ-level phenotyping accuracy and efficiency with UAV platforms, thereby accelerating the identification and utilization of crop germplasm resources.