Three-dimensional reconstruction and parameters extraction of walnut (Juglans regia L.) branches based on Neural Radiation Fields

Abstract

Three-dimensional (3D) reconstruction is important for obtaining morphological information and making intelligent management decisions for fruit trees. Thus, a method for the 3D reconstruction and parameters extraction of branches based on Neural Radiation Fields (NeRF) was proposed for walnut (Juglans regia L.) trees. This approach combined Structure from Motion (SfM) with NeRF and used multi-view images to reconstruct branches. First, a dataset of multi-view images of walnut trees was built and camera poses were obtained using SfM. Second, WalnutNeRF was optimized by incorporating hash encoding, piecewise sampler and appearance embedding features to address challenges associated with complex outdoor environments and accurately reconstruct branches. A scale recovery method using calibration objects was employed to extract branch parameters. The effectiveness of WalnutNeRF was evaluated by analyzing rendering performance, reconstruction efficiency, point cloud quality, and the accuracy of extracted branch parameters. WalnutNeRF outperformed existing methods in terms of the quality of rendered images and the accuracy of estimated depth, as determined using PSNR, SSIM, LPIPS, and other metrics. WalnutNeRF resulted in a branch reconstruction accuracy of 90.94 %, with a training time that was 9-time faster than that of SfM-MVS. Compared with SfM-MVS, WalnutNeRF decreased reconstruction errors for the main branches, lateral branches, and watershoots by 72 %, 67 %, and 57 %, respectively, and decreased the errors in length by 7.09 %, 4.33 %, and 65.07 %, respectively. Accordingly, WalnutNeRF decreased the reconstruction time, while increasing accuracy, providing robust support for the development of intelligent management applications (e.g., intelligent pruning) for walnut trees.