Language models that match reader experience are better predictors of reading times

Abstract

Humans differ in the language experience that they accumulate, due to differing interests, reading habits and profession. This experience can be expected to affect their linguistic expectations when reading texts from domains that are very familiar to them. The present article explores whether language models trained to match the experience of readers produce surprisal estimates that more accurately predict the reading times of those readers than the usually employed general language models. We use a German eye-tracking corpus of biology and physics students reading expository texts from these domains. We adapt a neural language model to the experience of these two groups of readers via two domain adaptation methods and varying amounts of training data. The evaluation against one early and two late reading measures suggests that aligning language models with the readers’ experience to predict the processing effort results in a better fit on late measures than using a model with a high linguistic accuracy. Our findings highlight the opportunities for exploring the cognitive plausibility of language models with respect to psychological constructs.