How the Multimodal Media in Augmented Reality Affects Museum Learning Experience

As AR is envisioned to become the leading yet paradigm-shifting user interface metaphor for situated computing, it is becoming essential to understand how users process and perceive AR and its enabling technologies since the effect of AR relied heavily on users’ perception to integrate digital information with the real world. From the perspective of situated cognition and learning, this evaluation study investigated how the integrative multimodal representations in AR affects visitors’ museum experiences. A genuine exhibit of citrus fruits in an experimental gallery was constructed using marker-based AR and video see-through displays. A between-subject experiment with 48 college students was conducted to evaluate the effects of integrative multimodal representations on users’ flow experiences and interaction behaviors. Preliminary findings suggested that users perceived the integrative multimodal representations in AR as more engaging, compared to the static exhibit. Participants regarded themselves to have clearer goals of visit toward the AR exhibit, and they appreciated the immediate feedback provided by AR for them to confirm their actions.