Review: The evolution of chemometrics coupled with near infrared spectroscopy for fruit quality evaluation. II. The rise of convolutional neural networks

Abstract

The Part 1 prequel to this review evaluated the evolution of modelling techniques used in evaluation of fruit quality over the past three decades and noted a progression towards the use of artificial neural networks (ANNs) and convolutional neural networks (CNNs). In this review, Part 2, the use of CNNs for NIR fruit quality evaluation is explored, given the success of CNNs in various other fields, such as image, video, speech, and audio processing, and the availability of large (open source) datasets of fruit spectra and reference quality attribute, which is required for the training of CNN models. The review provides an overview of deep learning and the CNN architectures and techniques used in NIR spectroscopy for regression modelling, with advantages and disadvantages identified. Studies using CNN for NIR based fruit quality evaluation are then critically examined. Eight publications have presented on models using the same open-source mango dry matter calibration and test set, enabling inter-method comparisons. CNN models have been demonstrated to be accurate, precise and robust. Techniques of transfer learning for CNN models offer an alternative solution to model updating and calibration transfer methods applied in traditional chemometrics. The review has highlighted crucial areas that require resolution and exploration in this application through future research, including, (i) data requirements for training a CNN (ii) optimal spectral pre-processing for CNN (iii) CNN architecture and hyper-parameter selection and tuning for fruit quality evaluation (iv) CNN model interpretability and explainability. Future studies must conduct clearer comparison to partial least squares (PLS) regression and shallow ANNs to better assess the prospective benefit of using CNN, a more complex model. The potential for visualisation of spectra relevance to the CNN model using techniques such as GradCam, currently employed in visualising 2D-CNN models, remains to be explored.