Supporting Remote Experimentation Through Computer-Mediated Communication

With the emergence of COVID-19 and the resultant reduction in direct personal contact, we were faced with a novel challenge of how to conduct studies that involved both semi-structured elicitation of detailed cognitive information and the evaluation of software interfaces and algorithms. We leveraged distributed communication technologies to support data collection across three separate human-computer interaction studies. We describe these studies in terms of specific computer-mediated communication methods and features and how we used them. We also present lessons learned, potential risks and detriments of using such approaches, (e.g., connectivity dependence, reduced ability to observe non-verbal cues, and diminished interpersonal interaction), along with how the research community can compensate for the shortcomings of distributed data collection in human-centered research.