GoNature AR: Air Quality & Noise Visualization Through a Multimodal and Interactive Augmented Reality Experience

As Extended Reality (XR) media experiences become a commodity, there is a unique opportunity to deploy XR for environmental awareness. Interaction challenges of Augmented Reality (AR) still exist, focused on limited gesture and head tracking affordances. AR technologies should also be seamlessly integrated with sensor data, analytics and ultimately status prediction, to be visualized in an AR experience, rather than merely superimposing visuals onto the real world. This paper presents an innovative, work-in-progress, multimodal AR experience integrating interactive narration, gestures, hands recognition and voice commands, while a citizen is wearing a head-worn AR display, promoting environmental awareness, health, and wellness. By combining AR technologies and a sensor network, GoNature AR provides citizen awareness of real-time, multimodal, air and noise pollution data.