Crop Yield Prediction Using Multimodal Meta-Transformer and Temporal Graph Neural Networks

Abstract

Crop yield prediction is a crucial task in agricultural science, involving the classification of potential yield into various levels. This is vital for both farmers and policymakers. The features considered for this task are diverse, including weather, soil, and historical yield data. Recently, plant images captured in different modalities, such as red–green–blue, infrared, and multispectral bands, have also been utilized. Most of these data are inherently temporal. Integrating such multimodal and temporal data is advantageous for yield classification. In this work, a deep learning framework based on meta-transformers and temporal graph neural networks has been proposed to achieve this goal. Meta-Transformers allow the modeling of multimodal interactions, while temporayel graph neural networks enable the utilization of time sequences. Experimental results on the publicly available EPFL multimodal dataset demonstrate that the proposed framework achieves a high classification accuracy of nearly 97%, surpassing other state-of-the-art models, such as long short-term memory networks, 1-D convolutional neural networks, and Transformers. In addition, the proposed model excels in accuracy metrics, with a precision of approximately 98%, an F1-Score of 91%, and a recall of 94% in crop yield prediction.