Control-Display Affordances in Simulation Based Education

Abstract

Mixed reality opens new ways of connecting users to virtual content. With simulation-based education and training (SBET), mixed reality offers an enriched environment to experience digital learning. In turn, learners can develop their mental models to process and connect 2D/3D information in real-world settings. This paper reports on the use of the Microsoft HoloLens to create a mixed reality SBET environment. The challenges of this investigation are harmonising augmented real-world content, including the use of real-time, low-latency tracking of tangible objects and the interaction of these with the augmented content. The research emphasis is on technology-mediated affordances. For example, what affordance does the HoloLens provide the leaner in terms of interactive manipulation or navigation in the virtual environment? We examine this through control-display (CD) gain in conjunction with cyber-physical systems (CPS) approaches. This work builds on previously attained knowledge from the creation of an AR application for vocational education and training (VET) of stonemasonry.