AI-Human Collaboration for in Situ Interactive Exploration of Behaviours From Immersive Environment

Abstract

Experiments in immersive environments allow the collection of large amounts of data that are closely related to individual behaviour. The recording of such experiments allows for the complex study of under-constrained tasks. That is, tasks that allow for a high degree of contingency in their resolution. This contingency allows for better discrimination of individual behaviour. However, the high complexity of the tasks makes them difficult to analyse. My thesis aims to discuss the advantages of Immersive Analytics for analysing hybrid sequential data (trajectory and events) generated in immersive environments. The analysis needs to be performed at a very high level of abstraction due to the high contingency of behaviours extracted from immersive environments. The massive amount of data generated highlights the need to build a model that allows feature extraction at a high level of abstraction. Since the exploration scheme is unknown in advance, the visualisations provided to the analyst should be highly interactive and adaptable to follow the analyst’s queries as he or she searches for new insights in the data.