Implementing Human-Robot Interaction Evaluation Using Standard Test Methods for Response Robots

Adam Norton1, Brian Flynn1, and Holly Yanco1 ABSTRACT The standard test methods developed through the ASTM E54.09 Committee on Homeland Security Applications; Response Robots have served the robotics community for over a decade and a half. The test methods specified through this committee provide a common framework, scalable structure, and malleable test apparatuses that can be used to exercise many different robotic capabilities. In recent years, the committee has emphasized how the test methods can be used for training purposes to evaluate operator proficiency. The human-robot interaction (HRI) characteristics of a robot system, including its control interface, are particularly influential to operator performance. The operator’s knowledge of the robot system and how to use it effectively will also affect performance. To this end, two potential standards efforts are proposed that can aid in characterizing and demonstrating effective HRI. The first is an expansion of an existing standard practice for recording robot configuration to include HRI characteristics and methods by which to measure the effectiveness of these characteristics. The second is a new standard practice that is aimed at demonstrating effective HRI by introducing variable test apparatus settings to elicit decision-making capabilities from the operator and system. The goal of this practice is to highlight HRI techniques to the standards community, such as utilizing assistive autonomous functionality for obstacle avoidance and error mitigation. By incorporating HRI evaluation in these ways, HRI can be seen as an integral component of a response robot system.