High-Resolution NIR Prediction from RGB Images: Application to Plant Phenotyping

Abstract

In contrast to the conventional RGB cameras, Near-infrared (NIR) spectroscopy provides images with rich information concerning the biological process of plants. However, NIR spectroscopy is a costly affair and produces low-resolution (LR) images. In this context, recently deep learning-based methods have been proposed in computer vision. In addition, the development of phenomics facilities has facilitated the generation of large plant data necessary for the utilization of these deep learning-based methods. Motivated by these developments, we propose a novel attention-based pix-to-pix generative adversarial network (GAN) followed by a super-resolution (SR) module to generate high-resolution (HR) NIR images from corresponding RGB images. An experiment including extraction of phenotypic data based on HR NIR images has also been conducted to evaluate its efficacy from an agricultural perspective. Our proposed architecture achieved state-of-the-art performance in terms of MRAE and RMSE on the Wheat plant multi-modality dataset.