Multimodal and Context-Aware Interaction in Augmented Reality for Active Assistance

Augmented reality eyewear devices (e.g. glasses, headsets) are poised to become ubiquitous in a similar way than smartphones, by providing a quicker and more convenient access to information. There is theoretically no limit to their applicative area and use cases and many of them are already explored such as medical, education, industry, entertainment, military? Some interactions to these eyewear devices are becoming a standard such as mid-air hand gestures and voice command. Paradoxically, nowadays, in many use cases where these kinds of eyewear devices are currently implemented, the users cannot perform these available interactions without constraints: e.g. when the users are already using their hands, when they are in a noisy environment or the opposite where silence is required and the vocal command could not be used properly, or even in a social context where both mid-air hand gestures and vocal command could be seen as weird or undesired for the users. Thus, this thesis project aims to extend interactivity of augmented reality eyewear devices: 1) by providing more discrete interaction such as head gesture based on cognitive image schemas theory, metaphorical extension and natural user interfaces based on the smart watch finger touch gesture, 2) by using the context of the user to provide the more convenient interface and feedback in the right space and time. The underlying objective of this project is to facilitate the acceptance and usage of augmented reality eyewear devices.