Towards Human Objective Real-Time Trust of Autonomy Measures for Combat Aviation

Abstract

ABSTRACT Objective Can objective trust indicators be measured and computed in real time in a within visual range aerial combat “dogfight” scenario consisting of a manned autonomous own-ship versus an adversary? Background Previous research has focused on human trust of automation during simulated ground combat and human trust of civil aviation automation. Those studies largely consisted of subjective measures of trust analyzed post-hoc. Human trust of autonomy in high-risk situations such as dogfighting had not yet been studied. Method Nine evaluation pilots participated in an experiment that consisted of nine operationally relevant live-flight vignettes during which they conducted a mission manager task while simultaneously monitoring an autonomous agent controlling their own-ship in an aerial dogfight. Eye tracking, objective real-time mental workload, and mission manager task performance were all recorded. Results Potential indicators of trust were objectively measured in real-time as a function of autonomy disengagement, human cross-check ratio of autonomy, and real-time objective mental workload, as validated by traditional subjective trust measures. Measured trust still needs to be categorized as appropriate trust, overtrust, or undertrust in future work. Conclusion This first-ever real-time measure of objective indicators of trust in operationally relevant live-flight paves the way for determining appropriate human trust of autonomy in high-risk situations such as dogfighting. These measures could have utility in high-risk manned-unmanned teaming applications such as working with robots and automated trading.