Visual to near-infrared image translation for precision agriculture operations using GANs and aerial images

The rapid growth of computer vision and artificial intelligence (AI) techniques, along with advancements in sensory systems and unmanned aerial vehicles (UAVs), have profoundly impacted various fields such as Precision Agriculture (PA). A core operation in PA for crop monitoring and yield improvement is the combination of visual and near-infrared (NIR) wavelengths using pixel-wise operations known as Vegetation Indices (VIs). However, deploying costly multi-spectral sensory systems limits the scalability of existing PA solutions. Towards this direction, Generative Adversarial Networks (GANs) can be employed for transforming visual images to near-infrared representations, enabling the utilization of affordable off-the-shelf visual sensors and reducing system cost and complexity. Nevertheless, existing GAN-based methods for spectral domain translation often are limited to colorization models that produce pseudo-realistic images, neglecting the crucial spectral characteristics of the target domain. These synthetic images are commonly evaluated based on their visual plausibility rather than the spectral characteristic’s consistency. In the context of precision agriculture, such translations from visual to NIR domain may lead to inaccurate VI calculations and unreliable vegetation health estimation, making the usability of the synthesized data questionable. To overcome these limitations, we propose a model-agnostic modification for GANs that leverages the semantic information of VIs in the translation process. Our approach introduces an additional branch to the GAN architecture, calculating the Normalized Difference Vegetation Index (NDVI) from the input RGB and the generated NIR image. By backpropagating the additional branch loss, our method enforces the network to produce meaningful NIR representations that accurately preserve the domain’s spectral characteristics. We deploy our approach on two widely used GAN architectures, Pix2Pix and CycleGAN, and evaluate the synthesized results on relevant datasets. Experimental results demonstrate that the proposed method provides accurate and meaningful translations of visual to NIR images. The synthesized images maintain their semantic context under the near-infrared spectral attributes, making them suitable for precise VI calculations, vegetation health estimation, and efficient utilization in relative precision agriculture applications.