‘Uncertainty audit’ for ecosystem accounting: Satellite-based ecosystem extent is biased without design-based area estimation and accuracy assessment

Abstract

There are currently no guidelines in the System of Environmental-Economic Accounting Ecosystem Accounting (SEEA EA) for quantifying and disclosing uncertainty. However, without quantifying uncertainty, it is unclear whether or not accounting tables contain biased (erroneous) area estimates which do not reflect real land cover changes. We use Oslo municipality in Norway as a case study to illustrate best practices in quantifying unbiased area estimates using design-based statistical methods. As input for ecosystem extent accounts, we compared a custom Sentinel-2 land cover map with a globally available one called Dynamic World for 2015, 2018 and 2021. The design-based area estimation involved (i) generating a stratified probability sample of locations using the satellite-based maps to define strata, (ii) assigning ecosystem type labels to the samples using photointerpretation according to a response design protocol, and (iii) applying a stratified area estimator to produce 95% confidence intervals around opening, closing and change stocks in the extent accounting table. We found that pixel counting practices, currently adopted by the SEEA EA community, led to biased extent accounts, particularly for ecosystem conversions, with biases averaging 195% of the true change value derived from design-based methods. We found that the uncertainty inherent in state-of-the-art satellite-based maps exceeded the ability to detect real change in extent for some ecosystem types including water and bare/artificial surfaces. In general, uncertainty in extent accounts is higher for ecosystem type conversion classes compared to stable classes, and higher for 3-yr compared to 6-yr accounting periods. Custom, locally calibrated satellite-based maps of ecosystem extent changes were more accurate (81% overall accuracy) than globally available Dynamic World maps (75%). We suggest that rigorous accuracy assessment in SEEA EA will ensure that ecosystem extent (and consequently condition and service) accounts are credible. A standard for auditing uncertainty in ecosystem accounts is needed.