Multispectral image reconstruction from RGB image for maize growth status monitoring based on window-adaptive spatial-spectral attention transformer

Abstract

Multispectral image analysis is an effective way to detect crop growth status. However, the complexity of manufacturing process and technology of multispectral image acquisition equipment make data acquisition expensive. Therefore, a method based on a window-adaptive spatial-spectral attention transformer is proposed to reconstruct multispectral images using RGB images of maize. First, RGB and hyperspectral images of the maize are obtained, and the reflectance data from classic and preferred band combinations are extracted from the hyperspectral image. Then, a transformer model is constructed to evaluate and compare the reconstruction efficacy of the 5-band and 10-band combinations across four attention modes: spatial, spectral, spatial-spectral, and window-adaptive spatial-spectral attention. The best-performing reconstruction results are selected and compared with the original data from three perspectives: image, spectrum, and model effect. The 10-band multispectral image reconstructed by the window-adaptive spatial-spectral attention mechanism is highly similar to the original image, with a reflectance correlation exceeding 0.99. Furthermore, its application in monitoring crop growth status (i.e., maize chlorophyll) yields results closely aligned with actual reflectance data: R_C² is 0.76, R_V² is 0.64, while RMSE_C and RMSE_V are 3.63 mg/L and 2.94 mg/L, respectively. To further explore the model performance, the new sensitive bands are selected to be reconstructed in the maize V7 stage. The results from the chlorophyll content prediction model are as: R_C² is 0.64, R_V² is 0.60, with RMSE_C and RMSE_V are 5.61 mg/L and 5.62 mg/L, respectively. Therefore, the window-adaptive spatial-spectral attention transformer can accurately reconstruct multispectral images and establish precise growth status monitoring models, providing technical support for low-cost field maize growth detection.