GRPE: High-fidelity 3D Gaussian reconstruction for plant entities

Abstract

3D Plant models hold significant importance for constructing virtual worlds. Currently, there is a lack of algorithms capable of achieving high-fidelity reconstruction of plant surfaces.

We propose a unified architecture to reconstruct high-fidelity 3D surface models and render realistic plant views, which enhances geometric accuracy during Gaussian densification and mesh extraction from 2D images.

The algorithm initially employs large vision models for semantic segmentation to extract plant objects from 2D RGB images, generating sparse mappings and camera poses. Subsequently, these images and point clouds are processed to produce Gaussian ellipsoids and 3D textured models, with the algorithm detecting smooth regions during densification. To ensure precise alignment of the Gaussians with object surfaces, the algorithm incorporates a robust 3D Gaussian splatting method that includes an outlier removal algorithm. Compared to traditional techniques, this approach yields models that are more accurate and exhibit less noise. Experimental results demonstrate that our method outperforms existing plant modeling approaches, surpassing existing methods

In terms of PSNR, LPIPS, and SSIM metrics. The high-precision annotated plant dataset and system code are available at https://github.com/DYH200009/GRPE.