Uncovering Representation Bias in Large-scale Cellular Phone-based Data: A Case Study in North Carolina

Abstract

Large cellular phone-based mobility datasets are an important new data source for research on human movement. We investigate and illustrate bias in representation in a large mobility data set at the census block group, tract, and county levels. We paired American Community Survey (ACS) 2019 data with SafeGraph (SG) cell phone mobility data to elucidate potential bias in SG data by examining ACS estimated population against the number of devices in the SG data, stratifying by key sociodemographic variables such as income, percent Black population, percent of population over 55 years, percent of population 18–65 years, percent of people living in crowded living conditions, and urbanization level. We evaluated whether the bias varied over time by examining a 10-month period. This bias changes with key demographic characteristics and changes over time. Specifically, we see underrepresentation in areas that have the highest percentage of Black population at all aggregation levels. We also see underrepresentation at all levels in areas with the highest percentage of working age residents as well as areas with the lowest median incomes. Researchers should be cautious when using mobility datasets because of bias differential on key sociodemographic factors and collection time.