Improving the estimation of the Boyce index using statistical smoothing methods for evaluating species distribution models with presence-only data

Species distribution models (SDMs) underpin a wide range of decisions concerning biodiversity. Although SDMs can be built using presence-only data, rigorous evaluation of these models remains challenging. One evaluation method is the Boyce index (BI), which uses the relative frequencies between presence sites and background sites within a series of bins or moving windows spanning the entire range of predicted values from the SDM. Obtaining accurate estimates of the BI using these methods relies upon having a large number of presences, which is often not feasible, particularly for rare or restricted species that are often the focus of modelling. Wider application of the BI requires a method that can accurately and reliably estimate the BI using small numbers of presence records. In this study, we investigated the effectiveness of five statistical smoothing methods (i.e. thin plate regression splines, cubic regression splines, B-splines, P-splines and adaptive smoothers) and the mean of these five methods (denoted as ‘mean') to estimate the BI. We simulated 600 species with varying prevalence and built distribution models using random forest and Maxent methods. For training data, we used two levels for the number of presences (NP_train: 20 and 500), along with 2 × NP_train and 10000 random points (i.e. random background sites) for each modelling method. We used the number of presences at four levels (NP_bi: 1000, 200, 50 and 10) to investigate its effect, together with 5000 random points to calculate the BI. Our results indicate that the BI estimates from the binning and moving window methods are severely affected by the decrease of NP_bi, but all the estimates of the BI from smoothing-based methods were almost always unbiased for realistic situations. Hence, we recommend these methods for estimating the BI for evaluating SDMs when verified absence data are unavailable.