AI for Toggling the Linearity of Interactions in AR

Abstract

Interaction in augmented reality (AR) or mixed reality environments is generally classified into two modalities: linear (relative to object) or non-linear (relative to camera). Switching between these modes tailors the AR experience to different scenarios. Such interactions can be arduous in cases when on-board touch interaction is limited or restricted as is often the case in medical or industrial applications that require sterility. To solve this, we present Sound-to-Experience where the modality can be effectively toggled by noise or sound which is detected using a modern Artificial Intelligence deep-network classifier.