Field-based phenotyping for poplar seedlings biomass evaluation based on zero-shot segmentation with multimodal UAV images

Poplar trees are widely cultivated for their ecological and economic benefits. Studying the phenotypes of poplar seedlings can enable the selection of optimal cultivation methods to enhance yield and quality. UAV-based low-altitude remote sensing with optical sensors captures images and spectral data for such studies. However, deep learning in UAV plant phenotyping faces the challenge of requiring substantial time and effort to label image samples for model training. This paper aims to assess the efficiency of using Grounding DINO-SAM2 for zero-shot instance segmentation of individual poplar seedlings across multiple genotypes. An automatic program calculates image features from RGB and multispectral mask areas, including canopy projection, color, texture, and spectral reflectance, which are then used to establish a biomass estimation model based on two years of data. The study obtained the following results: (1) The Grounding DINO-SAM2 model was used to implement zero-labelled sample instance segmentation of 400 image data. After modifying the sample with incorrect target recognition quantity in less than 15 min, the total model took only 0.5 h, with a precision of 0.943, which greatly saved time and computing cost compared with mainstream fully-supervised segmentation models. (2) A poplar seedling biomass estimation model based on multimodal image features was established. After comparing and optimizing single-sensor and multi-sensor combined with different modelling algorithms, it was found that the CNN test set accuracy (R²) reached 0.823. This research provides a lightweight, cost-effective approach for plant image segmentation and feature extraction, promoting advances in intelligent management and monitoring for agriculture and forestry.