A model-based approach for multi-device user interactions

Abstract

This paper presents an approach for modeling multi-device user interactions, based on task models. We use Concur-TaskTrees (CTTs) as a domain-specific language, which we extend here by a labeling mechanism to model multi-device interactive applications. While CTTs are used to specify temporal and causal relations between tasks, we add operators to specify the device mapping in a flexible and expressive way. The main novelty is the introduction of the two new operators, Any and All, to specify if a task should be executed on any or on all of a set of devices. We show that this is applicable in scenarios of connected, smart devices where a task can be executed on a multitude of devices. We present formal semantics for our extension of CTTs as well as a tool chain based on the Qt toolkit for generating code for distributed UIs. This includes a mapping from high-level tasks to concrete UI controls and a distributed execution model based on state machines. The new concepts are validated in several case studies.