Behavioral Experiments Online?

Abstract

Online experiments offer several advantages over traditional laboratory experiments. However, for reaction time experiments, precise stimulus presentation and response detection is crucial. The precision of online experiments could be compromised due to increased variance arising from varying hardware configurations among participants, lack of control over experimental conditions, and the absence of an examiner. In this study, we conducted an online experiment using the avatar-Simon task to investigate whether small differences in reaction times could be examined using online experiments conducted with the experimental toolkit PsyToolkit. In the avatar-Simon task, participants respond to the color of vertically presented stimuli in front of avatars by pressing a left or right button. Reactions are faster when the position of the stimulus, defined from the avatar's point of view, matches the position of the response. Compared to the previous laboratory experiment, we observed lower effect sizes and more timeouts but were able to replicate the avatar-Simon effect overall. Based on further distributional and reliability analyses, PsyToolkit appears to be suitable tool to detect behavioral effects in the range of tens of milliseconds. We discuss differences and similarities with the original laboratory study and suggest how to address potential problems associated with online experiments.