Multimodal spectral-textual alignment and fusion of large language model and small model for seed quality assessment: A case study in maize vigor detection

Abstract

Maize seed vigor is crucial for agricultural productivity, directly influencing crop yield. This study presents a novel fusion framework that aligns semantic features from a large language model (LLM) with spectral features for maize vigor detection. We propose a collaborative strategy where an LLM guides the pre-training of an encoder-decoder (ED) model, which is then fine-tuned on spectral datasets. The fine-tuned ED model (FTED) achieves mean accuracies of 0.9488 and 0.9762 on datasets labeled with Seed Vigor Index (SVI) and Triphenyl Tetrazolium Chloride content (TTCc), respectively. For spectral reconstruction, it yielded mean squared errors of 0.0013 and 0.0039. Notably, embryo spectra are highly sensitive to vigor, while whole seed spectra maximize detection accuracy. The FTED model demonstrates robust generalization across diverse datasets and resilience to simulated spectral noise. By using LLM capabilities to enhance a lightweight ED model, this approach offers an efficient, scalable solution for vigor detection with potential applications in agricultural domains.