“All Right, Mr. DeMille, I’m Ready for My Closeup:” Adding Meaning to User Actions from Video for Immersive Analytics

While the use of machine learning and computer vision to classify human behavior has grown into a large, well-established, interdisciplinary area of research, one area that is somewhat overlooked is the intersection of computer vision as a tool for evaluating user behavior in Virtual Reality, particularly in the context of immersive analytics and visualization. We draw on the literature from pattern recognition, computer vision, and machine learning to compose a simple, comparatively resource-cheap pipeline for camera-based extraction of features of professional analyst users and of their sessions in an existing VR visualization system, ImAxes. Our results show high accuracy in predicting self-reported features of the users, even as survey responses about user experience with the immersive interface are somewhat ambiguous in varying based on these features.