Evaluation of cognitive load and user experience in alternative interaction modes under different noise degrees

Abstract

In the context of multimodal interaction, user-centered research on alternative interaction modes is crucial for their application in real-world scenarios. In intelligent cockpits of specialized vehicles and aircraft, broadband continuous noise is a common challenge. This study aims to investigate the cognitive load and user experience associated with alternative interaction modes when performing tasks under varying levels of broadband continuous noise. 24 participants completed a point-and-select task with four interaction modes: Touch-Based Interaction (TBI), Speech-Based Interaction (SBI) with Speech Recognition (SRT) and Wizard of Oz (WoZ), Gesture-Based Interaction (GBI), and Multimodal Interaction (MMI) at three noise levels (45dBA, 65dBA, 85dBA). Cognitive load was assessed through blinks, pupil diameters, and NASA-TLX scores, while task performance (completion time, error rate) and user experience (pragmatic quality (PQ), hedonic quality (HQ), and attractiveness) were recorded. Results showed that speech recognition errors in noisy environments increased cognitive load and decreased user experience for SBI-SRT and MMI-SRT. Regardless of noise degrees, SBI-WoZ exhibited the lowest cognitive load, followed by GBI and MMI-WoZ. TBI had the highest cognitive load. GBI required the most physical demand and effort. TBI and GBI showed better robustness in noise, achieving higher HQ and PQ, while HQ and PQ for SBI and MMI declined with noise. MMI provided a better user experience than SBI. SBI-SRT was seen as redundant at 85 dBA. These findings provide valuable insights for the practical application of alternative interaction modes in noisy environments.