{"title":"Unbiased evaluation of social vulnerability: A multimethod approach using machine learning and nonparametric statistics","authors":"Hiroki Yokoyama , Yoshiyasu Takefuji","doi":"10.1016/j.cities.2025.106519","DOIUrl":null,"url":null,"abstract":"<div><div>This paper introduces a globally applicable bias-aware framework for interpreting machine-learning feature importances by benchmarking them against classical statistics. Using CDC's Social Vulnerability Index data, we compare five predictive models—both nonlinear and linear—with three ground-truth association measures. While nonlinear models deliver superior accuracy, their importance scores systematically inherit and amplify biases from feature correlations and imbalance—a universal concern for ML interpretability. We demonstrate that key vulnerability drivers are robustly detected only when statistical validation accompanies model explanations. This research contributes methodological advances to algorithmic interpretability knowledge and offers international policy recommendations: implement statistical validation protocols for high-stakes ML applications, utilize complementary approaches for robust feature assessment, and establish global standards for interpretability in vulnerable population analytics. These findings generalize across diverse contexts where transparent, bias-resilient feature ranking drives equitable decision-making.</div></div>","PeriodicalId":48405,"journal":{"name":"Cities","volume":"168 ","pages":"Article 106519"},"PeriodicalIF":6.6000,"publicationDate":"2025-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cities","FirstCategoryId":"96","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0264275125008224","RegionNum":1,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"URBAN STUDIES","Score":null,"Total":0}
引用次数: 0
Abstract
This paper introduces a globally applicable bias-aware framework for interpreting machine-learning feature importances by benchmarking them against classical statistics. Using CDC's Social Vulnerability Index data, we compare five predictive models—both nonlinear and linear—with three ground-truth association measures. While nonlinear models deliver superior accuracy, their importance scores systematically inherit and amplify biases from feature correlations and imbalance—a universal concern for ML interpretability. We demonstrate that key vulnerability drivers are robustly detected only when statistical validation accompanies model explanations. This research contributes methodological advances to algorithmic interpretability knowledge and offers international policy recommendations: implement statistical validation protocols for high-stakes ML applications, utilize complementary approaches for robust feature assessment, and establish global standards for interpretability in vulnerable population analytics. These findings generalize across diverse contexts where transparent, bias-resilient feature ranking drives equitable decision-making.
期刊介绍:
Cities offers a comprehensive range of articles on all aspects of urban policy. It provides an international and interdisciplinary platform for the exchange of ideas and information between urban planners and policy makers from national and local government, non-government organizations, academia and consultancy. The primary aims of the journal are to analyse and assess past and present urban development and management as a reflection of effective, ineffective and non-existent planning policies; and the promotion of the implementation of appropriate urban policies in both the developed and the developing world.