Investigating the Effect of Sensor Data Visualization Variances in Virtual Reality

This paper investigates the effect of real-time sensor data variances on humans performing straightforward assembly tasks in a Virtual Reality-based (VR-based) training system. A VR-based training system has been developed to transfer color and depth images, and constructs colored point clouds data to represent objects in real-time. Various parameters that affect sensor data acquisition and visualization of remotely operated robots in the real-world are varied. Afterward, the associated task performance is observed. Experimental results from 12 participants performed a total of 95 VR-guided puzzle assembly tasks demonstrated that a combination of low resolution and uncolored points has the most significant effect on participants’ performance. Participants mentioned that they needed to rely upon tactile feedback when the perceptual feedback was minimal. The most insignificant parameter determined was the resolution of the data representations, which, when varied within the experimental bounds, only resulted in a 5% average change in completion time. Participants also indicated in surveys that they felt their performance had improved and frustration was reduced when provided with color information of the scene.