A multi-temporal multi-spectral attention-augmented deep convolution neural network with contrastive learning for crop yield prediction

Abstract

Precise yield prediction is essential for agricultural sustainability and food security. However, climate change complicates accurate yield prediction by affecting major factors such as weather conditions, soil fertility, and farm management systems. Advances in technology have played an essential role in overcoming these challenges by leveraging satellite monitoring and data analysis for precise yield estimation. Current methods rely on spatio-temporal data for predicting crop yield, but they often struggle with multi-spectral data, which is crucial for evaluating crop health and growth patterns. To resolve this challenge, we propose a novel Multi-Temporal Multi-Spectral Yield prediction Network, MTMS-YieldNet, that integrates spectral data with spatio–temporal information to effectively capture the correlations and dependencies between them. While existing methods that rely on pre-trained models trained on general visual data, MTMS-YieldNet utilizes contrastive learning for feature discrimination during pre-training, focusing on capturing spatial–spectral patterns and spatio–temporal dependencies from remote sensing data. Contrastive learning finds the relative features that distinguish crop growth patterns over different temporal intervals. The stacked attention mechanism is applied to improve spectral–spatial feature extraction by focusing on the most important spectral bands and spatial regions, further enhancing forecasting precision. We use an optimization approach inspired by natural balance processes to identify key spectral and temporal features for effective feature selection in crop yield prediction. We evaluate MTMS-YieldNet on various datasets using remote sensing images from Sentinel-1, Sentinel-2, and Landsat-8, treating each source as a distinct dataset to capture diverse agricultural patterns. Both quantitative and qualitative assessments highlight the excellence of the proposed MTMS-YieldNet over seven existing state-of-the-art methods. For the Sentinel-1 dataset, the MTMS-YieldNet achieves 0.336 MAPE, 0.497 RMSLE, and 0.362 SMAPE, and for the Landsat-8 dataset, it achieves 0.353 MAPE, 0.511 RMSLE, and 0.428 SMAPE. On Sentinel-2, it achieves an outstanding performance of 0.331 MAPE, 0.589 RMSLE, and 0.433 SMAPE, demonstrating its effectiveness in yield prediction across varying climatic and seasonal conditions in this agriculturally significant region. The outstanding performance of MTMS-YieldNet improves yield predictions and provides valuable insights that can assist farmers in making better decisions, potentially improving crop yields.