Busy beeway: a game for testing human-automation collaboration for navigation

Abstract

This study presents Busy Beeway, a mobile game platform to investigate human-automation collaboration in dynamic environments. In Busy Beeway, users collaborate with automation to evade stochastically moving obstacles and reach a series of goals, in game levels of increasing difficulty. We are motivated by the need for reliable navigation aids in stochastic, dynamic environments, which are highly relevant for self-driving vehicles, UAVs, underwater and surface vehicles, and other applications. The proposed mobile game platform is agnostic to the particular algorithm underlying the autonomous system, can be used to evaluate both fully autonomous as well as human-in-the-loop systems, and is easily deployable, for large, remote user studies. This last element is key for rigorous study of human factors in navigation aids. Through a small 32--user study, we evaluate preliminary findings regarding the relative efficacy of collaborative and fully autonomous navigation, the relationship between success rate and users' learned trust in the automation (gathered via pre- and post-experiment surveys), and tolerance to error (for decisions made by the automation and by the user). This study validates the feasibility of Busy Beeway as a platform for human subject studies on human-automation collaboration, and suggests directions for future research in human-aided planning in difficult environments.