Simulating lexical decision times with large language models to supplement megastudies and crowdsourcing.

Abstract

Megastudies and crowdsourcing studies are a rich source of information for word recognition research because they provide processing times for thousands of words. However, the high cost makes it impossible to include all words of interest and all relevant participant groups. This study explores the potential of fine-tuned large language models (LLMs) to generate lexical decision times (RTs) similar to those of humans. Building on recent findings that LLMs can accurately estimate word features, we fine-tuned GPT-4o mini with 3000 words from a megastudy. We then gave the model the task of generating RT estimates for the remaining words in the dataset. Our findings showed a high correlation between AI-generated and observed RTs. We discuss three applications: (1) estimating missing RT data, where AI can fill in gaps for words missing in some megastudies, (2) verifying results of virtual experiments, where AI-generated data can provide an additional layer of validation for results of virtual experiments, and (3) optimizing human data collection, as researchers can run simulations before conducting studies with humans. While AI-generated RTs are not a replacement for human data, they have the potential to increase the flexibility and efficiency of megastudy research.