ShadowClones: an Interface to Maintain a Multiple Sense of Body-space Coordination in Multiple Visual Perspectives

Abstract

In this paper, we propose ShadowClones, an interface that supports interactions in which a single user can interact with multiple bodies in multiple spaces. Recent teleoperation technologies have allowed a user controlling multiple objects simultaneously, but at the same time, it also exhibited a significant challenge, which can be attributed to the high cognitive load caused by switching and recogning various spaces/perspectives repeatedly and instantly. To tackle this challenge, by taking advantage of pre-attentive visual cues for users’ simultaneous information processing, we designed and evaluated a new user interface, called Shadow Clones, that projects self-body information in unattended areas for increasing the awareness of body-space relationships and allowing users to seamlessly switch across different visual perspectives from avatars or remote robots. We then explored the proposed approach through a simple visual reaching task with a performance evaluation in terms of task completion time and success rate. The results showed superior performance when compared with a condition that presents no projections of users’ body movements in unattended areas. We conclude by discussing possible mechanisms of this enhancement as well as two potential scenarios using the shadow clones approach, including new entertainment content for virtual reality e-sports and multiple robot teleoperation such as in a construction site or a disaster site, without compromising operational performance.