An integrated framework for multi-commodity agricultural price forecasting and anomaly detection using attention-boosted models

Abstract

Global agricultural systems are increasingly exposed to price instability driven by climate extremes, logistic disruptions, and market uncertainty. These conditions complicate efforts to monitor and manage price behaviours in essential commodity markets. Micro, small, and medium enterprises (MSMEs), which operate with constrained resources and limited access to data-driven tools, are particularly susceptible to sudden and irregular price shifts. Their ability to maintain stable operations depends on timely identification of market anomalies and reliable planning information. This underscores the importance of accurate price forecasting, yet deep learning models such as Bidirectional Long Short-Term Memory (LSTM) and the Gated Recurrent Unit often struggle to capture long-term dependencies and detect irregular price behaviors. To bridge the gap, this study proposes a deep learning framework that integrates Transformer models for price prediction and an attention-boosted LSTM Variational Autoencoder (VAE) for anomaly detection. Using daily price data collected from the period of January 2020 to mid-June 2024, this study demonstrated that Transformers outperformed traditional models while accurately capturing market trends and sudden fluctuations. Additionally, the attention-boosted anomaly detection model can outperform standard LSTM and artificial neural network-VAEs in identifying unexpected price changes. The proposed models outperformed baseline methods by achieving lower forecasting and anomaly detection errors. By addressing critical limitations in existing forecasting approaches, specifically their inability to capture abrupt anomalies, this study provides essential support for enhancing MSMEs’ resilience and improving decision-making under volatile market conditions.