Inversion of on-tree peach firmness via high-fidelity fruit finite element models and sim-to-real deep transfer learning

Firmness is an important attribute of peaches related to texture and ripeness. The inversion of on-tree peach firmness contributes to monitoring fruit ripeness and determining the optimal harvest time for enhancing fruit quality. However, accurate inversion of fruit firmness is still challenging because of the limited number, range and distribution of training datasets. Therefore, an acoustic vibration method based on high-fidelity fruit finite element (FE) models and sim-to-real deep transfer learning was proposed for the inversion of the on-tree peach firmness with limited samples. Ten FE models of peaches comprising skin, flesh, pits and kernels were constructed to generate simulated vibration responses of on-tree peaches during fruit ripening. A 1D-Inception-SE network was established to extract features from the vibration responses to characterise peach firmness. To reduce the difference in feature distribution between the simulated dataset and the experimental dataset, a deep transfer learning method based on a domain adversarial neural network was introduced. The results indicated that the high-fidelity FE models of peaches were reliable for simulating the vibration behaviours of on-tree peaches during fruit ripening. The proposed deep transfer learning method raised the R² of the test set from 0.79 to 0.83. Compared to the 1D-Inception-SE model with 320 experimental samples, the deep transfer learning model with both simulated data and 160 experimental samples achieves superior performance while requiring only half the number of experimental samples. The results demonstrated that the proposed method could efficiently and effectively improve the firmness inversion accuracy for on-tree peaches by measuring limited samples.