Enhanced feed-forward for a user aware multi-touch device

Abstract

Common multi-touch devices guide the user with feedback visualization during or after a registered interaction. Feed-forward techniques are less frequently used or not common at all. Our approach aims at a continuous process in which the system is aware of the users before, during, and after an explicit interaction takes place. This opens up the possibility for novel scenarios of user centered applications. Our setup utilizes Microsofts's depth-camera Kinect to collect the user's posture data in combination with a multi-touch device. This is a low cost and easy to install approach for collecting detailed information about the people and their position in close proximity of a multi-touch table as well as the location of their physical contact. Based on this information, we propose five phases of interaction and analyze the sequence of input during a typical workflow. Eight application concepts show the relevance of these phases using appropriate forms of visualization and we evaluated three of those concepts in a user-study.