Using Bayesian analysis to quantify and reduce uncertainty in experimental measurements — A narrow-angle radiometer case study

Abstract

Formal uncertainty analysis is an important but sometimes overlooked component of experimental work. Without quantified uncertainty, it is difficult to draw definitive conclusions from the experimental data, as a lack of formal uncertainty analysis leaves the reliability of the data unknown. An added benefit to performing uncertainty analysis is that once uncertainty is quantified, steps may be taken to mitigate it. There are two layers of uncertainty in experimental measurements: uncertainty due to sources present during calibration of the measurement device (“calibration-scenario uncertainty”) and additional uncertainty due to sources present during the experimental measurement (“experimental-scenario,” or total, uncertainty). We formalize a generic protocol (the “Bayesian Uncertainty Quantification and Reduction Protocol”) for use in any experimental measurement to first quantify and then strategically refine error sources in the data. In this work, we utilize a method of Bayesian uncertainty quantification developed and presented by Spinti et al. (2021). Once the uncertainty is measured, the protocol targets the largest contributors to the uncertainty; the experimentalists may iterate the relevant steps of the protocol to refine these error sources until the uncertainty is either below a desired threshold or they reach the physical limits of the system.

We illustrate the practical use of the protocol with radiometric intensity data taken in an industrial-scale power plant. First, we calculate the calibration-scenario uncertainty of the intensity data. Next, we modify the calibration procedure and the instrument model, which reduced the calibration-scenario uncertainty (2$\sigma$) from 21.5% to 2.81% (an 87% reduction). Lastly, we utilize this quantified uncertainty with replicate data at the experimental scale to estimate the total or experimental-scenario uncertainty of the quantity of interest: time-averaged intensity measurements in the industrial boiler. Our reduction in calibration-scenario uncertainty reduces the estimated total uncertainty of the intensity measurements by roughly one-third. Despite these reductions, the total uncertainty remains high. We recommend reapplying the protocol using data from a future experimental campaign, coupled with a high-fidelity model of the boiler, to address the high total uncertainty in these measurements.