Intelligent sorting of pecan shelled products using hyperspectral fingerprints and deep learning

Post-harvest processing of tree nuts is an essential process that enhances their quality and economic value. Currently air lathe and handpicking are the prevailing methods used in the industry for sorting shelling products. However, the air lathe approach is inaccurate because it requires further handpicking of the remaining shell fragments, which is labor-intensive, subjective, and time-consuming. The aim of this paper was to explore the potential of visible near-infrared (VNIR) and near-infrared (NIR) hyperspectral imaging systems (HSI) to accurately classify pecan shelled products into three classes (“shells,” “inner-wall,” and “kernels”). The VNIR (400–1000 nm) and NIR (900–1700 nm) systems were used to acquire hyperspectral images. The extracted spectral data were used to develop four machine learning classifiers (Decision Tree (DT), Gradient Boosting (GB), Random Forest (RF), and Support vector machine (SVM)), and deep learning methods (convolutional neural network (CNN), hybrid CNN combined with long short-term memory (LSTM), and CNN-CNN-LSTM. Among the machine learning classifiers, the SVM achieved superior accuracies of 95.81%, and 96.91% for VNIR and NIR spectral data, respectively. The hybrid CNN-LSTM achieved an accuracy of 97.17% and 98.36% for VNIR and NIR spectra data, respectively, while the fused spectral developed on CNN-CNN-LSTM yielded the superior result of 99.29% among all the models. The results obtained in this study demonstrated the high potential of adopting HSI systems for the classification of pecan shelled products for intelligent sorting in the pecan processing industry.