Calibration transfer of sugar content prediction models for agricultural products via NIR spectral augmentation and reconstruction architecture

Abstract

Rapid non-destructive determination of sugar content in agricultural products such as pear and sugarcane is critical for optimising harvest timing and enhancing market competitiveness. However, near infrared spectroscopy-based prediction models suffer from degraded generalisation performance during cross-year or cross-device transfers, primarily due to insufficient sample sizes and spectral variability, which significantly hinders their large-scale deployment in agricultural engineering. To address these challenges, this study proposes a novel spectral masked autoencoder generative adversarial network (SMAEGAN), which integrates adversarial training with a masked autoencoder strategy to synthesise high-fidelity spectral data with enhanced cross-domain generalisation capabilities, thereby effectively mitigating data scarcity in few-shot transfer learning scenarios. Furthermore, a pyramid associative self-attention spectral transformer (PASASpTr) was introduced that hierarchically extracts multi-scale spectral features through an innovative pyramid architecture, enabling the synergistic capture of both local band-specific variations and global patterns. Comprehensive experimental evaluations demonstrate that PASASpTr significantly outperforms traditional convolutional neural networks in sugarcane sugar content prediction, achieving a root mean square error (RMSE) of 3.40, a coefficient of determination (R²) of 0.93, and a parameter size of 0.94 M. When combined with SMAEGAN-augmented reconstructed spectra for transfer learning, an average RMSE reduction of 44.68 % was achieved in the sugarcane dataset collected across different devices, and an average RMSE reduction of 40.65 % was achieved in the pear dataset collected across different years and production lines. This methodological architecture significantly reduces the reliance on target-domain annotated samples, while offering a lightweight and scalable solution for predicting sugar content across a wide range of agricultural products.