Does data collection mode matter for word learning trajectory: processing variability in lab and web environments

Critics have expressed concerns about reaction time (RT) data collected online due to their sensitivity to timing accuracy (Chetverikov & Upravitelev, 2016). While research involving first-language speakers has largely demonstrated the comparability of web and lab RT data, second-language (L2) researchers have only recently started to do so (Patterson & Nicklin, 2023). Further, there is a lack of research on whether learning trajectories captured by RTs in-person studies may be generalized to the web environment. This study compares how L2 word learning progressed as indexed by RTs and the coefficient of variation (CV) online and in person. Eighty-three first-language speakers of English studied 20 Swahili words and then went through 20 rounds of animacy judgment tasks. Participants completed the experiment in a lab or online. We found markedly different trends for RTs and CVs in lab and web environments. In general, results from the lab largely replicated previous in-person studies on the same topic: RTs declined significantly, and CVs followed an inverted U-shape before a plateau. For the web participants, RTs showed smaller changes while CVs were stable throughout learning. We discussed the viability of conducting learning experiments online where the outcome is assessed by RT-based measures.