Online Interactive Experiments on Networks

Abstract

Conducting human experiments using crowdsourcing platforms, such as Amazon Mechanical Turk, has made it possible to collect a much larger amount of experimental data in a much shorter period of time relative to what was possible in traditional physical lab settings. This has provided a new suite of methods for conducting randomized experiments in socio-technical systems, allowing for straightforward causal inference [1-4]. However, using crowdsourcing platforms to experimentally study real-time interactions between individuals presents numerous practical challenges. These studies need fairly large groups of subjects to be present simultaneously in each session, and outcomes typically occur at the level of the group (i.e., session) rather than the individual. Yet most crowdsourcing platforms are not designed to facilitate simultaneous structured interactions between subjects. Thus, it can be difficult (and expensive) to recruit enough participants to achieve a sufficient degree of statistical power (especially for session-level outcomes). In this tutorial, we will discuss best practices for designing and conducting online social network experiments where human subjects (and programmed bots) interact simultaneously within a specified network structure. We will show how the experimental design can be informed by computational models in an iterative process (i.e., using experimental data to calibrate the computational model and use the computational model to optimize the design and find the right parameters for the experiments). We will also introduce additional tools/platforms that facilitate conducting such studies and walk the audience through the implementation steps of a typical experiment on networks using customized and publicly available software.