Towards a behavior-oriented specification and testing language for multimodal applications

Initiated by the ubiquity of mobile devices, human computer interaction has evolved beyond the classic PCs' mouse and keyboard setup. Smartphones and tablets introduced new interaction modalities to the mass market and created the need for specialized software engineering methods. While more and more powerful SDKs are released to develop interactive applications, specifying user interaction is still ambiguous and error-prone, causing software defects as well as misunderstandings and frustration among project team members and stakeholders. We present an approach addressing this problems by demonstrating how to incorporate multimodal interaction into user acceptance tests written in near-natural language using Gherkin and formal gesture descriptions.