Leveraging VR Techniques for Efficient Exploration and Interaction in Large and Complex AR Space with Clipped and Small FOV AR Display

Abstract

In this paper, we propose to take advantage of the digital twinned environment to interact more efficiently in the large and complex AR space in spite of the limited sized and clipped FOV of the AR display. Using the digital twin of the target environment, “magical” VR interaction techniques can be applied, as visualized and overlaid through the small window, while still maintaining the spatial association to the augmented real world. First we consider the use of amplified movement within the corresponding VR twinned space to help the user search, plan, navigate and explore efficiently by providing an effectively larger view and thereby better spatial understanding of the same AR space with less amount of physical movements. Secondly, we also apply the amplified movement and in addition, the stretchable arm to interact with relatively large objects (or largely spaced objects) which cannot be seen in their entirety at a time with the small FOV glass. The results of the experiment with the proposed methods have showed advantages with regards to the interaction performance as the scene became more complex and task more difficult. The work illustrates the concept of and potential for XR based interaction where the user can leverage the advantages of both VR and AR mode operations.