Speech augmented multitouch interaction patterns

Abstract

Touch- and voice-based input have emerged as the most popular and relevant interaction modes to enable a natural interaction with computer systems. However, until now, they have mostly been treated separately. In particular, explicit design knowledge on the effective combinations of these modes for an improved user experience is currently not available in a comprehensive form. In this paper, we address this shortage and introduce design patterns which support developers in exploiting the possibilities of combined voice and touch interaction for newly developed systems, so that interaction with these systems becomes more natural for the respective end users.