Interact: Tightly-coupling Multimodal Dialog with an Interactive Virtual Assistant

Abstract

Interact is a mobile virtual assistant that uses multimodal dialog to enable an interactive concierge experience over multiple application domains including hotel, restaurants, events, and TV search. Interact demonstrates how multi- modal interaction combined with conversational dialog en- ables a richer and more natural user experience. This demonstration will highlight incremental recognition and under- standing, multimodal speech and gesture input, context track- ing over multiple simultaneous domains, and the use of multimodal interface techniques to enable disambiguation of erors and online personalization.