Quantifying expert opinion in the UK water industry: an experimental study

Abstract

This paper reports an experiment in which 32 experts at three water companies quantified their opinions about two problems: the cost of refurbishing specified pumping stations and the length of unrecorded S24 sewers in four towns. Medians and both unconditional and conditional assessments of quantiles were elicited and assessments were compared with reality and given scores. The quantiles that experts were asked to assess varied and the experts differed in the extent of their relevant background knowledge. The influences of these factors on standard deviations and scores are examined. The main focus is on modelling methods and ways of using the elicited assessments to form subjective probability distributions. We consider fitting log-normal distributions to model asymmetric distributions and also examine models to relate a quantity's assessed standard deviation and interquantile range to its size, finding that interquantile ranges can be modelled more accurately. In addition, different approaches to separating opinions about components of variance are evaluated and compared.