Simulated responses from large language models are weak proxies for human economic preferences across 74 countries.

Abstract

Large-scale surveys are central to investigating economic preferences but are costly and slow. Can large language model (LLM)-simulated responses substitute for human survey data? We conducted a virtual Global Preference Survey, simulating six experimentally validated economic preferences for 67,201 matched human respondents across 74 countries. Across 20 LLM configurations with diverse prompts and robustness checks, we found systematic but modest individual-level alignment (87.5% of Spearman correlations have 95% confidence intervals (CIs) above zero), with LLMs recovering the direction of associations in 77.8% (95% CI, (71.6%, 83.1%); CI-based consistency ratio) of gender, age and cognitive-ability dimensions, but only 48.1% (95% CI, (45.1%, 51.2%)) of cultural dimensions. Ablation and error decompositions suggest weaker alignment when removing demographic information than when removing country labels, and in many models alignment is weaker in economically undeveloped and less digitally connected countries. LLM-simulated responses may recover some individual- and group-level patterns but are not yet reliable proxies for cross-national representative surveys in macro-level inference or policy evaluation.