A Bootstrap-assisted Methodology for the Estimation of Prediction Uncertainty in Multilayer Perceptron-based Calibration

Abstract

Background

In calibration, analytical figures of merit (AFOMs) are statistical parameters of great importance for method validation. In recent decades, relevant contributions have been made to estimate AFOMs in many calibration scenarios. However, calculating AFOMs in nonlinear models, like those based on artificial neural networks (ANNs), is still a matter of investigation. In this work, the problem of estimating the prediction uncertainty quantified by the variance ($$) associated with the prediction of a test sample, in the context of multilayer perceptron (MLP)-based calibration was tackled.

Results

Two well-established statistical techniques, i.e., the delta method and the bootstrap, were combined to develop a methodology for variance estimation. Besides, the errors coming from both concentration and spectral variables were taken into account for model formulation. The proof of concept was based on a 95% confidence interval coverage analysis calculated from multiple simulated nonlinear calibration datasets. The results showed that the delta method is suitable for determining a general variance structure for a nonlinear calibration model, considering errors from both concentrations and instrumental signals. Likewise, the bootstrap has proven to be a powerful tool for estimating model variability, particularly due to its ability to bypass the need for explicit formula derivation, even in the presence of the flexibility that characterizes the MLP.

Significance

The proposed strategy was applied to two already published nonlinear experimental datasets modeled by MLP, where the prediction uncertainty was assessed for the first time. This work represents a novel step toward fully characterising ANN-based calibration models. This is urgently needed to improve the analytical results report and facilitate the transfer of new analytical methodologies to the industry.