Exploration of Usage Patterns for Multimodal Input Interactions

Abstract

This work attempts to study the usage patterns for multimodal input interactions like, pattern of errors, user's modality preferences when dealing with errors, their pattern of switching across modalities and their preferences of input modalities. In order to study those aspects of user behavior, we analyzed the usage data of the eighteen participants in our experiment which involved usage of input modalities like keyboard, speech, touch and pen, as multimodal input combinations. From the usage patterns of the participants, we analyzed that the user's choice of modality for error recovery and overall modality preferences differs for navigation and editing tasks. However, their other usage patterns do not significantly differ based on the type of task at hand. Our results showed that the majority of the errors while using the speech input were due to system errors, while those for keyboard and touch were user committed errors. Some of these findings would be useful in designing user interactions with better multimodal input integration for various tasks.