Intelligent Sensory Modality Selection for Electronic Supportive Devices

Abstract

Humans operating in stressful environments, such as in military or emergency first-responder roles, are subject to high sensory input loads and must often switch their attention between different modalities. Conventional supportive devices that assist users in such situations typically provide information using a single, static sensory modality; however, this carries the risk of overload when the modalities for the primary task and the supportive device overlap. Effective feedback modality selection is essential in order to avoid such a risk. One potential method for accomplishing this is to intelligently select the supportive device's feedback modality based on the user's environment and given task; however, this may result in delayed or lost information due to the performance cost resulting from switching attention from one modality to another. This paper describes the design and results of a human-participant study designed to evaluate the benefits and risks of various intelligent modality-selection strategies. Our findings suggest complex interactions between strategies, sensory input load levels and feedback modalities, with numerous significant effects across many different performance metrics.