A hyperspectral and RGB crop pests/diseases image fusion method based on SADAE-GAN with combined perceptual gradient loss

Abstract

Accurate monitoring of crop pests and diseases is essential due to their diversity, widespread impact, and increasing severity. However, RGB images lack spectral richness, and hyperspectral images suffer from resolution loss caused by long exposure times. Existing fusion methods often lose important features, reducing the accuracy and practicality of pest and disease monitoring. To address these issues, this paper proposes a SADAE-GAN model, which combines Deep Convolutional Generative Adversarial Networks (DCGAN), Self-Attention (SA), and Denoising Auto-Encoders (DAE) techniques. The model enhances both local and global feature extraction through a primary processing module and DAE in the generator, while the discriminator uses an SA attention mechanism to improve image reconstruction quality. Additionally, a novel perceptual gradient loss function and an Autoencoder and Continuous Wavelet Transform (Auto-CWT) image transformation method are introduced to optimize image generation and monitoring performance. Experimental results across multiple datasets demonstrate that the proposed SADAE-GAN outperforms existing popular fusion methods like CycleGAN, ResNet, SSRNET, InfoGAN, and QIS-GAN. For evaluation metrics SSIM, PSNR, MSE, and IS, the maximum performance improvement reached 20.7 %, 68.2 %, 39.7 %, and 33.0 %, respectively. In addition, ablation studies confirm the effectiveness of the proposed model, and robustness tests further reveal the model’s strong practical applicability. These results demonstrate that SADAE-GAN excels in fusion quality, robust feature preservation, and strong generalization capability, providing a promising and efficient solution for precise agricultural pest and disease monitoring.