Assessment of light environment conditions for reconstruction of multispectral images by conditional adversarial networks

Abstract

Understanding vegetation through its reflectance in the visible and near-infrared spectrum is vital for gaining biophysical and structural insights about vegetation. However, the spectral reflectance on meaningful bands (e.g., red-edge, near-infrared) is not always available because of the camera's spectral response restrictions. In this context, previous research addresses the lack of multispectral information by reconstructing it using deep-learning approaches. Although there are promising outcomes, the influence of varying illumination conditions on this process still needs to be explored. Thus, this work examines if conditional Generative Adversarial Networks (cGANs) can infer environment illumination for achieving an appropriate multispectral image reconstruction. In particular, the spectral reconstruction performance of cGANs is investigated under six different scenarios with different illumination (occurring over a whole day), focusing on green, red-edge, and near-infrared bands. Note that the dataset used for this research has become publicly available. These results indicated that illumination conditions influenced the performance of cGAN models in generating spectral images. Specifically, the cGANs could not infer the source image illumination to output a reliable reconstructed spectral image. Furthermore, although including samples under different illumination improved cGANs' performance, the generated multispectral images tended to be darker than actual images.