Comparing the multimodal interaction technique design of MINT with NiMMiT

Abstract

With new sensors that can capture hand and body movements in 3D, novel interaction techniques gain importance. But development of new forms of interaction is highly iterative, depends on extensive user testing and therefore is expensive. We propose a model-based notation using statecharts and mappings to ease multimodal interaction technique design. This model-based specification can be used to communicate designs, for evaluation and to enable re-use. Our contribution continues previous research on model-based interaction technique design considers multimodal interaction and addresses problems like the state explosion, error management and consideration of output modalities mentioned by earlier research. We evaluate our notation by comparing it with NiMMiT referring to the same use case to identify similarities, strength and problems.