The impact of map accuracy on area estimation with remotely sensed data within the stratified random sampling design

One of the core applications of satellite-based classification maps is area estimation. Regardless of the algorithms used, maps will always contain errors stemming from imperfect input and training/calibration data, incomplete data coverage, and spectral and/or temporal confusion between land cover and land use classes. Because of omission and commission errors, the pixel-counting area estimator will be a biased estimator for area estimation. Therefore, the remote sensing research and application communities have developed a framework and recommended practices to address this problem. One such approach is a stratified random sampling design, in which classification maps could be used for stratification in the sampling design, and areas are estimated from the sample data, which represent reference data or reference class labels. As such, the quality of the map, i.e., producer's (PA) and user's accuracy (UA), will not affect the bias of the estimator, as the bias depends on the sampling design and the choice of estimator. However, map quality will impact the efficiency of stratification: a more accurate map will require a smaller sample size to reach the target variance of the estimate, or it will yield improved precision if the sample size is fixed. This study aims to provide a quantitative assessment of the impact of map accuracies on area estimation within the stratified random sampling design. The relative bias of the pixel-counting estimator is expressed using class-specific PA and UA, and shown to be $\frac{\mathrm{PA}}{\mathrm{UA}}-1$. Furthermore, for the case of binary classification, elements of the confusion matrix, as well as the sample size, variance of the area estimator, and relative efficiency of stratification (the ratio of the products of variance and sample size for the two sampling approaches) are expressed using PA and UA. Numerical simulations demonstrate how relative efficiency depends on area estimation objectives, target area proportion, and the map's performance metrics (PA and UA). Such dependence is nonlinear, and the impact of those parameters varies. For example, when the target class is minor or rare (i.e., its true proportion is <<0.5), the impact of PA outweighs that of UA. As the target area proportion increases, the impact of accuracies converges, and UA has a greater impact on efficiency than PA. There are multiple values in the PA/UA space, though constrained, to reach the same objectives, e.g., in terms of relative efficiency, sample size, and target variance. Overall, this study offers map producers a criterion that can be used to benchmark algorithm performance for map generation when area estimation is the primary objective of the classification maps.