Detection of the number of wheat stems using multi-view images from smart glasses

The number of stems in wheat populations is a fundamental parameter to achieve high yields and a critical agronomic trait in wheat production and variety selection. Although smart agricultural technology can estimate various agronomic parameters, the wheat stem is often obscured by multiple canopy leaves, making estimation challenging. Consequently, the current method to determine the stem number predominantly relies on labor-intensive manual techniques, which are inefficient and significantly influenced by subjective factors. This study proposes the use of augmented reality (AR) glasses as an imaging data acquisition tool to detect the number of wheat stems with high precision based on features from the top canopy and lateral images of wheat clusters. Following a correlation analysis, four color features, Coverage, the texture feature Contrast, and two lateral peak features SI (Peaks1 and Peaks2) of the top canopy image were identified. The study comparatively analyzed the image features from three perspectives for their accuracy in detecting the number of wheat stems. The results indicated a strong correlation between the peak feature (SI) and the number of wheat stems with an R² value above 0.75. The estimation using only canopy image features (CC) resulted in significant errors, where the RMSE was 20 under high-density planting conditions. Using only Peaks1 and Peaks2 yielded higher accuracy in the stem estimation, but uncertainties persisted in some high-density scenarios. Furthermore, the study combined CC and SI for the estimation and used a random forest algorithm to construct a stem estimation model. This model maintained an RMSE below 10, even under high planting densities and below 5 under low densities, which demonstrated high accuracy. This study could provide insights into stem detection for crops similar to wheat and offer a reference for other studies that require hands-free and first-person perspective image acquisition.