Sentence processing by humans and machines: Large language models as a tool to better understand human reading.

Abstract

Online measures of reading have been studied with the goal of understanding how humans process language incrementally as they progress through a text. A focus of this research has been on pinpointing how the context of a word influences its processing. Quantitatively measuring the effects of context has proven difficult but with advances in artificial intelligence, large language models (LLMs) are more capable of generating humanlike language, drawing solely on information about the probabilistic relationships of units of language (e.g., words) occurring together. LLMs can be used to estimate the probability of any word in the model's vocabulary occurring as the next word in a given context. These next-word probabilities can be used in the calculation of information theoretic metrics, such as entropy and surprisal, which can be assessed as measures of word-by-word processing load. This is done by analyzing whether entropy and surprisal derived from language models predict variance in online measures of human reading comprehension (e.g., eye-movement, self-paced reading, ERP data). The present review synthesizes empirical findings on this topic and evaluates their methodological and theoretical implications.