The concurrent use of touch and mid-air gestures or floor mat interaction on a public display

This paper investigates a novel approach to simultaneously use the qualities of touch and mid-air gestures or floor mat interaction on a public display. We demonstrate that although the concurrent use of multiple interaction modalities appeared functionally possible and has the potential to augment a single display with both personal and public functionalities, it is hampered by issues relating to display and interaction blindness, in addition to social discomfort and what we propose to name affordance blindness. We describe this blindness as the inability to understand the interaction modalities of a public display. Overall, passers-by tend to assume that a public display only supports a single interaction modality, an issue that is hard to overcome with ergonomic or visual interventions, or via physical curiosity objects. Although user engagement was relatively limited, we believe that our qualitative results highlight several crucial design and usability aspects when designing multi-modal, interactive public displays. Furthermore, this study demonstrates that previous knowledge on public display interaction cannot be simply ported to other contexts, and that more diverse content-types or contextual use cases should be evaluated "in-the-wild" to attain more generalizable insights.