Investigating spatial-temporal bias of LLMs

Abstract

Large Language Models (LLMs) are emerging as powerful knowledge and expert systems with notable capabilities in understanding and inferring various intelligent tasks. However, their spatiotemporal cognition biases remain largely underexplored, despite being highly consequential for effectively leveraging LLMs to power diverse applications in understanding, explaining, and forecasting such tasks. In light of this, this paper presents an investigation of the presence and patterns of spatiotemporal bias in LLMs. Specifically, this paper first constructs two datasets from the perspectives of economic and social forecasting, each paired with corresponding model-predicted values for the same spatiotemporal scope across four different LLMs. Then, a novel autocorrelation measurement approach is introduced, alongside a set of quantification methods, to jointly evaluate correlation in biases across both space and time. The results show notable variation in performance and bias across models and tasks, with uncommon and more sensitive tasks exhibiting worse performance, and certain LLMs producing regionally clustered errors while others exhibit near-random distributions. Out of all other methods of changing prompts, incorporating temporal context significantly improves predictive accuracy, particularly for volatile or low-frequency events. Overall, these findings highlight the partial but inconsistent internalization of real-world spatiotemporal patterns in LLMs, and the proposed methods provide tools for quantifying and interpreting spatiotemporal bias, thereby offering guidance for designing fairer and more reliable LLM-based expert systems and applications.