Comparing sustained attention performance across laboratory-based versus web-based settings

Abstract

Recently, there have been many attempts to replicate effects that have traditionally been observed in laboratory-based settings using web-based settings. While many classic effects have indeed been found to replicate, it is also true that web-based experiments often yield slower reaction times and less accurate responses compared with lab-based experiments. The aim of the current study was to compare performance on the Sustained Attention to Response Task (SART) across lab-based and web-based settings using a repeated-measures, within-subjects design. The results indicated no significant differences between the lab-based and web-based settings in reaction times for both go and no-go trials, as well as in post-error slowing. Additionally, no significant differences were observed in the accuracy of no-go trials. However, the standard deviation of reaction times was smaller in the lab-based setting, and accuracy in go trials was higher in the lab-based setting compared with the web-based setting. These findings suggest that a web-based setting can effectively replicate a laboratory-based setting in terms of reaction times, indicating that sustained attention effects are comparable across settings. However, the lower accuracy observed in the web-based setting suggests that participants may engage with the task less carefully. Furthermore, the greater variability in reaction times in the web-based setting implies a higher susceptibility to external influences. These results highlight the importance of careful methodological considerations when designing and interpreting the results of web-based experiments.