Usability Evaluation of Warning Cues for Remotely Supervised Autonomous Agricultural Machines.

Abstract

Highlights: Compared visual-auditory and visual-tactile modalities for AAM supervision. Both modalities are highly usable, with preferences shaped by farming experience. Non-farming participants responded faster to visual-auditory cues. User background influenced warning design effectiveness and interface needs.

Abstract: As autonomous agricultural machines (AAMs) become more prevalent, effective communication through warning systems is essential for safe and efficient remote supervision. This study evaluated the usability and cue effectiveness of two bimodal warning modalities (visual-auditory (VA) and visual-tactile (VT)) during a simulated remote supervision task. Usability was assessed from the supervisor's perspective using the Computer System Usability Questionnaire (CSUQ). Cue effectiveness was examined through objective performance-based measures aligned with three situation awareness levels: (1) noticeability and response time as measures of perception, (2) accuracy and comprehension time as measures of comprehension, and (3) accuracy and projection time as measures of projection. Thirty participants (5 with farming experience and 25 without farming experience) interacted with a simulated human-machine interface (HMI). Both modalities received high usability ratings, although the visual-tactile was rated more favorably by farming participants. Projection accuracy was significantly lower for farming participants using visual-auditory cues (86.7%), χ²(3, N = 540) = 5.37, p = .147. Although comprehension accuracy differed slightly across modalities-particularly among farming participants-these differences were minimal and not statistically significant. In contrast, non-farming participants exhibited significantly faster saccadic and manual response times to visual-auditory warnings, t(24) = 3.16, p = .004 and t(24) = 3.56, p = .002, respectively. Comprehension and projection times did not differ significantly by modality. Subjective preferences aligned with performance trends; non-farming participants favored visual-auditory cues, while farming participants preferred visual-tactile cues. Correlation analyses indicated that faster comprehension strongly predicted higher usability, and task accuracy was perfectly correlated with usability ratings. These findings underscore the influence of user experience on multimodal warning efficacy and support the need for adaptive, user-informed interface designs in AAM supervision.