Performance effects of multi-sensory displays in virtual teleoperation environments

Multi-sensory displays provide information to users through multiple senses, not only through visuals. They can be designed for the purpose of creating a more-natural interface for users or reducing the cognitive load of a visual-only display. However, because multi-sensory displays are often application-specific, the general advantages of multi-sensory displays over visual-only displays are not yet well understood. Moreover, the optimal amount of information that can be perceived through multi-sensory displays without making them more cognitively demanding than a visual-only displays is also not yet clear. Last, the effects of using redundant feedback across senses on multi-sensory displays have not been fully explored. To shed some light on these issues, this study evaluates the effects of increasing the amount of multi-sensory feedback on an interface, specifically in a virtual teleoperation context. While objective data showed that increasing the number of senses in the interface from two to three led to an improvement in performance, subjective feedback indicated that multi-sensory interfaces with redundant feedback may impose an extra cognitive burden on users.