Designwise: Design principles for multimodal interfaces with augmented reality in internet of things-enabled smart regions

Abstract

Technological developments, such as mobile augmented reality (MAR) and Internet of Things (IoT) devices, have expanded available data and interaction modalities for mobile applications. This development enables intuitive data presentation and provides real-time insights into the user’s context. Due to the proliferation of available IoT data sources, user interfaces (UIs) have become complex and diversified, while mobile devices have limited screen spaces. This state increases the necessity of design principles that help to secure sufficient user experience (UX). We found that studies of design principles for IoT-enabled MAR applications are limited. Therefore, we conducted a systematic literature review to identify existing design principles applicable to IoT-enabled MAR applications. From the state-of-the-art research, we compiled and categorized 26 existing design principles into seven categories. We analyzed the UIs of three IoT-enabled MAR applications with the identified design principles and user feedback gathered from each application’s evaluation to understand what design principles can be considered in designing these applications. Among the 26 principles, we find eight principles that are commonly identified as possible improvements for the applications based on their purposes. We demonstrate the practical use of the identified principles by redesigning the UIs, and we propose five new design principles derived from the application analysis. As a result, we summarized a total of 31 design principles, including the five new ones. We expect that our findings will give insight into the UX/UI design of IoT-enabled MAR applications for researchers, educators, and practitioners interested in UX/UI development.