Rapid and non-destructive detection of hollow defects in pecans by near-infrared spectroscopy combined with multimodal data fusion and deep learning

Abstract

Hollow defects seriously affect the quality and economic benefits of pecans, necessitating the removal of defective pecans from normal batches. In this study, near-infrared spectroscopy (NIRS) combined with multimodal data fusion and deep learning was proposed to detect hollow defects and their severity in pecans. Initially, different spectral preprocessing methods were applied and the preprocessed spectral data were fused to improve detection accuracy. Additionally, the physical parameters of pecans were incorporated to construct a “spectral features–physical parameters” multimodal fusion dataset. And traditional machine learning and deep learning methods were employed to develop binary and ternary classification models for detecting hollow defects in pecans. The results indicate that multimodal fusion significantly improves model performance. The optimized SVM model achieves overall accuracies of 94.19% and 88.37% for binary and ternary classifications, Deep learning models further enhances detection accuracy, with the CNN-MLP dual-stream model based on multimodal data fusion yielding the best results. The optimal binary classification model obtained an overall accuracy of 97.67%, while the ternary classification model achieved an accuracy of 90.7%. These results demonstrate that the CNN-MLP dual-stream model combined with multimodal data fusion can effectively improve the detection accuracy of hollow defects in pecans, providing a reliable and non-destructive detection method for pecan quality evaluation.