Analysing a UI’s Impact on the Usability of Hands-free Interaction on Smart Glasses

As smart glasses and other head-mounted devices (HMD) are becoming more developed, the number of different use cases and settings where they are deployed have also increased. This includes scenarios where the hands of the user are not available to interact with a system running on such hardware, which precludes some interaction designs from these devices, such as free-hand gestures or the use of a touchpad attached to the device (e.g., on the frame). Alternative modalities include head gestures and speech-based input. However, while these interfaces leave the hands of their users free, they are not as intuitive: common metaphors like touching, pointing, or clicking do not apply. Hence there is an increased need to explain these mechanisms to the user and to make sure they can be used to operate such a device. However, there is no work available on how this should be done properly.In the research presented here, we conducted a study on different ways to support the use of head gestures and voice control on HMDs. For each modality, an abstract as well as an explicit UI design for communicating their usage to users were designed and evaluated in a care setting, where hands-free interaction is necessary to interact with patients and for hygienic reasons. First results from a within-subjects analysis show that surprisingly there does not seem to be much of a difference in performance when comparing these approaches to each other as well as when comparing them to a baseline implementation which offered no additional help. User preferences between the designs diverged: participants often had one clear favourite for the head-gesture UIs while barely noticing the difference between the speech-based UIs. Preferences on certain designs did not seem to impact performance in objective and subjective measures such as error rates and questionnaire results. This suggests that either implementations’ support for these modalities should adapt to individual preferences or that there is a need to focus on other areas of support to increase usability.