NeRF-LAI: A hybrid method combining neural radiance field and gap-fraction theory for deriving effective leaf area index of corn and soybean using multi-angle UAV images

Abstract

Methods based on upward canopy gap fractions are widely employed to measure in-situ effective LAI ($L_e$) as an alternative to destructive sampling. However, these measurements are limited to point-level and are not practical for scaling up to larger areas. To address the point-to-landscape gap, this study introduces an innovative approach, named NeRF-LAI, for corn and soybean $L_e$ estimation that combines gap-fraction theory with the neural radiance field (NeRF) technology, an emerging neural network-based method for implicitly representing 3D scenes using multi-angle 2D images. The trained NeRF-LAI can render downward photorealistic hemispherical depth images from an arbitrary viewpoint in the 3D scene, and then calculate gap fractions to estimate $L_e$. To investigate the intrinsic difference between upward and downward gaps estimations, initial tests on virtual corn fields demonstrated that the downward $L_e$ matches well with the upward $L_e$, and the viewpoint height is insensitive to $L_e$ estimation for a homogeneous field. Furthermore, we conducted intensive real-world experiments at controlled plots and farmer-managed fields to test the effectiveness and transferability of NeRF-LAI in real-world scenarios, where multi-angle UAV oblique images from different phenological stages were collected for corn and soybeans. Results showed the NeRF-LAI is able to render photorealistic synthetic images with an average peak signal-to-noise ratio (PSNR) of 18.94 for the controlled corn plots and 19.10 for the controlled soybean plots. We further explored three methods to estimate $L_e$ from calculated gap fractions: the 57.5° method, the five-ring-based method, and the cell-based method. Among these, the cell-based method achieved the best performance, with the r² ranging from 0.674 to 0.780 and RRMSE ranging from 1.95 % to 5.58 %. The $L_e$ estimates are sensitive to viewpoint height in heterogeneous fields due to the difference in the observable foliage volume, but they exhibit less sensitivity to relatively homogeneous fields. Additionally, the cross-site testing for pixel-level LAI mapping showed the NeRF-LAI significantly outperforms the VI-based models, with a small variation of RMSE (0.71 to 0.95 m²/m²) for spatial resolution from 0.5 m to 2.0 m. This study extends the application of gap fraction-based $L_e$ estimation from a discrete point scale to a continuous field scale by leveraging implicit 3D neural representations learned by NeRF. The NeRF-LAI method can map $L_e$ from raw multi-angle 2D images without prior information, offering a potential alternative to the traditional in-situ plant canopy analyzer with a more flexible and efficient solution.