ARAPID: Towards Integrating Crowdsourced Playtesting into the Game Development Environment

Abstract

Iterative playtesting of games is crucial, but remains a time consuming and ad-hoc process. In this work, we streamlined and systematized the playtesting process by adding support for crowdsourced, on-demand playtesting directly into a game development environment to test and evaluate platform games. Our system, called ARAPID (As Rapid As Possible Iterative Design), manages playtesting from a control panel in Unity and recruits players from Mechanical Turk. With this system, game designers can launch playtests and visualize results directly in the editor. To better understand and evaluate this approach, we ran three studies on two basic platform games. First, a game designer used ARAPID to playtest and rapidly iterate on the design of the games; we found the games were improved toward specific design goals. Second, we analyzed the impact of different forms of recruitment on playtesting speed; we found that pre-recruitment and over-recruitment of players can reduce the time needed to run playtests. Third, we analyzed the impact of players' familiarity with the game on the precision of playtest results; we found that whether or not players are allowed to replay the game can impact results.