Evaluating species distribution model predictions through time against paleozoological records

Abstract

Species distribution models (SDMs) are widely used to project how species distributions may vary over time, particularly in response climate change. Although the fit of such models to current distributions is regularly enumerated, SDMs are rarely tested across longer time spans to gauge their actual performance under environmental change. Here, we utilise paleozoological presence/absence records to independently assess the predictive accuracy of SDMs through time. To illustrate the approach, we focused on modelling the Holocene distribution of the hartebeest, Alcelaphus buselaphus, a widespread savannah-adapted African antelope. We applied various modelling algorithms to three occurrence datasets, including a point dataset from online repositories and two range maps representing current and ‘natural’ (i.e. hypothetical assuming no human impact) distributions. We compared conventional model evaluation metrics which assess fit to current distributions (i.e. True Skill Statistic, TSS_c, and Area Under the Curve, AUC_c) to analogous ‘paleometrics’ for past distributions (i.e. TSS_p, AUC_p, and in addition Boyce_p, F2-score_p and Sorensen_p). Our findings reveal only a weak correlation between the ranking of conventional metrics and paleometrics, suggesting that the models most effectively capturing present-day distributions may not be the most reliable to hindcast historical distributions, and that the choice of input data and modelling algorithm both significantly influences environmental suitability predictions and SDM performance. We thus advocate assessment of model performance using paleometrics, particularly those capturing the correct prediction of presences, such as F2-score_p or Sorensen_p, due to the potential unreliability of absence data in paleozoological records. By integrating archaeological and paleontological records into the assessment of alternative models' ability to project shifts in species distributions over time, we are likely to enhance our understanding of environmental constraints on species distributions.