Nighttime environment enables robust field-based high-throughput plant phenotyping: A system platform and a case study on rice

Abstract

Plant phenotyping has emerged as a cornerstone for deciphering the complex interplay between plant genetics and environmental factors. To acquire reliable plant phenotypes, it is important to have an accurate, robust, and high-throughput plant phenotyping system platform. During the last decade, the community seems to have a consensus that such systems should be deployed and work in daytime. Many phenotyping results, however, can be inaccurate and unstable in daytime due to rapid changes in lighting and shadows, particularly for vision-based systems. In this work, we build upon a commercial vision-based high-throughput plant phenotyping (HTPP) platform TraitDiscover and customize a nighttime working mode for the platform. In particular, we incorporate several hardware designs tailored to the nighttime environment such as the array-style lighting equipment and the three-axis high-precision automated control system. On the software side, we also integrate state-of-the-art YOLOv8 object detection and K-Net semantic segmentation frameworks to enable high-performance nighttime image analysis. The feasibility and robustness of the system are demonstrated with a case study on rice. To quantify the effectiveness of phenotyping, a high-quality nighttime rice image segmentation dataset is collected, with 360 finely annotated masks of rice plants. Experimental results show that our customized system is able to achieve surprisingly high segmentation performance up to 93.52% mask IoU, which is significantly higher than the metrics reported from daytime phenotyping. From the mage analysis results, we further extract and validate 28 phenotyping parameters related to color, morphology, and texture status. The average $R^2$ between the inferred phenotype parameters and the actual values reached 0.95, demonstrating the reliability and robustness of the system in nighttime phenotyping. Our results and findings may encourage phenotyping practitioners to rethink the current de facto choice of deploying ‘daytime plant phenotyping platforms’.