How Accurate Are High Resolution Settlement Maps at Predicting Population Counts in Data Scarce Settings?

Despite the recent milestone of the world population surpassing 8 billion, disparities in population data reliability persist, with many countries facing outdated or incomplete census data. Such inaccuracies have far-reaching implications for various sectors, including public health, urban planning, and resource allocation. The study leverages the rich data environment provided by the detailed 2018 Colombian census data and its coverage indicator, which create a high-quality controlled environment to assess the performance of census-independent population estimation approaches. Drawing from a diverse range of environmental landscapes in Colombia, we evaluate the effectiveness of satellite imagery-derived settlement maps in conjunction with various modeling techniques. We explore two estimation approaches based on settlement maps: a data-driven machine learning approach exemplified by a random forest model and a process-driven probabilistic approach exemplified by a hierarchical Bayesian model. Our findings underscore the efficacy of Bayesian modeling in addressing data scarcity and bias, providing robust estimates and quantifying model uncertainty. However, the random forest model performs better when data inputs are detailed and unbiased. We further emphasize the importance of considering settlement map characteristics in the modeling process, while recognizing the overall limitations of relying solely on satellite imagery for population counts. Through a rigorous evaluation of different stages of the population modeling pipeline—data input, model selection, and outcome assessment—this study provides key insights into the challenges and requirements of using satellite imagery-derived settlement maps for population estimation in data-scarce contexts.