Integrating speech and two-dimensional gesture input-a study of redundancy between modes

Abstract

It is widely recognised that multiple modes of input give higher recognition rates than single modes. The use of multiple modes, such as speech and gesture, can allow for more natural human-computer interaction. Opportunities for error and ambiguity resolution also occur, as the same information may be available from more than one source, providing redundant or repeated information. This paper presents a study of the redundant integration of speech and 2D gesture input in a relatively simple, inexpensive multimodal input system. Empirical results about multimodal recognition rates at particular levels of redundancy are presented, and implications for the integration of multiple modes are discussed.