Managing the Complexity of Collaborative Simulation-Based Human-in-the-Loop Experimentation

Human-in-the-Loop (HITL) experimentation can be effective for assessing the efficacy of multi-person operations. Based on the experiment objectives of such operations, an experiment team creates a virtual experimentation environment in which multiple human subjects can collaborate to achieve mission objectives. Afterwards, analysts assess the effectiveness of new technologies or procedures under test. While there is much value in conducting simulation-based HITL experimentation, there is also a large degree of complexity. This paper presents a framework for managing the complexity of executing such an experiment by dividing the experiment team into several smaller specialized teams that collaborate using processes described in this paper. An innovation leadership team, scenario team, technical team, and Data Collection and Analysis (DCA) team work together to plan and execute the experiment and assess results. An experiment team can use the methodology presented here to manage complexity and, ultimately, accomplish the objectives of collaborative simulation-based HITL experiments.