Uncertainty of predictions in absorption spectroscopy: Modelling with quantile regression forest

Abstract

Machine learning modelling is becoming popular for estimating agricultural and environmental properties from their infrared spectra. Commonly in modelling with machine learning and in commercial software applications, however, uncertainty estimates of the prediction are seldom reported. Uncertainty quantification of variables predicted with infrared spectroscopy is yet highly relevant in a number of applications, such as in uncertainty propagation analyses studies or for drug exposure detection. In this paper, we report on the development and application of quantile regression forest to predict properties from infrared spectroscopic data along with a sample-specific estimate of the uncertainty. Quantile regression forest is a machine learning algorithm that builds on random forest and provides estimate of the mean but also of the full conditional distribution of the predicted variable. We illustrate the algorithm with two chemometric applications and evaluate the modelling approach for its ability for predict the variable of interest and quantify the uncertainty. Evaluation involved usual validation statistics but also the validation of the uncertainty with the prediction interval coverage probability calculated for various interval widths. We tested prediction and prediction uncertainty quantification of two soil properties (cation exchange capacity and total organic carbon) as well as the dry matter of mango. The results confirm the potential of quantile regression forests for prediction and uncertainty quantification of properties predicted from infrared spectroscopy data. In all cases, the predictions were accurate and sample-specific estimates of the uncertainty were obtained. Validation of the uncertainty showed that the interval width was too large, thus overestimating the uncertainty for most intervals. Nevertheless, we recommend its use for operational applications as well as in future software developments, in particular when the data inferred by the spectroscopic model are used in other applications.