Touching the Cloud: Bimanual annotation of immersive point clouds

Abstract

In this paper we present “Touching the Cloud”, a bi-manual user interface for the interaction, selection and annotation of immersive point cloud data. With minimal instrumentation, the setup allows a user in an immersive head-mounted display (HMD) environment to naturally interact with point clouds. By tracking the user's hands using an OpenNI sensor and displaying them in the virtual environment (VE), the user can touch the virtual 3D point cloud in midair and transform it with pinch gestures inspired by smartphone-based interaction. In addition, by triggering voice- or button-press-activated commands, the user can select, segment and annotate the immersive point cloud, thereby creating hierarchical exploded view models.