Detailed validation of large-scale Sentinel-2-based forest disturbance maps across Germany

Abstract

Monitoring forest areas with satellite data has become a vital tool to derive information on disturbances in European forests at large scales. An extensive validation of generated maps is essential to evaluate their potential and limitations in detecting various disturbance patterns. Here, we present the validation results of forest disturbance maps generated for four study areas in Germany using Sentinel-2 data from 2018 to 2022. We apply a time series filtering method to map annual forest disturbances larger than 0.1 ha based on spectral clustering and annual change magnitude. The presented method is part of a research study to design a precursor for a national German forest disturbance monitoring system. In this context, annual forest change areas are used to estimate affected timber volume and related economic losses. To better understand the thematic accuracies and the reliability of the area estimates, we performed an independent and extensive validation of the annual product using 20 validation sets embedded in our four study areas and comprising a total of 11 019 sample points. The collected reference datasets are based on an expert interpretation of high-resolution aerial and satellite imagery, including information on the dominant tree species, disturbance cause, and disturbance severity level. Our forest disturbance map achieves an overall accuracy of 99.1 ± 0.1% in separating disturbed from undisturbed forest. This is mainly indicative of the accuracy for undisturbed forest, as that class covers 97.2% of the total forest area. For the disturbed forest class, the user’s accuracy is 84.4 ± 2.0% and producer’s accuracy is 85.1 ± 3.4% for 2018 to 2022. The similar user’s and producer’s accuracies indicate that the total disturbance area is estimated accurately. However, for 2022, we observe an overestimation of the total disturbance extent, which we attribute to the high drought stress in that year leading to false detections, especially around forest edges. The accuracy varies widely among validation sets and seems related to the disturbance cause, the disturbance severity, and the disturbance patch size. User’s accuracies range from 31.0 ± 8.4% to 98.8 ± 1.3%, while producer’s accuracies range from 60.5 ± 37.3% to 100.0 ± 0.0% across the validation sets. These variations highlight that the accuracy of a single local validation set is not representative of a region with a large diversity of disturbance patterns, such as Germany. This emphasizes the need to assess the accuracies of large-scale disturbance products in as many different study areas as possible, to cover different patch sizes, disturbance severities, and disturbance causes.