Proposal of a Hand Motion and Control Method Matching System for Interaction Optimized for Mixed Reality Control: Focusing on Meta Interface and Augmented Behavior

Abstract

In an era where non-face-to-face meetings become common, eXtended Reality(XR) is rapidly developing and filling areas that are not satisfied in online meetings based on existing photos/video method. In particular, general users are also able to easily access and use HMD-type mixed reality devices. However, the basic operations applied in HMD-type in Mixed Reality(MR) with hands as the main input tool do not have a standardized system, and each manufacturer operates in a separate response to each other's hand movements. Therefore, this study considered that a systematic hand motion matching system considering the usability and efficiency of operations performed in mixed reality was necessary, and conducted a study to clarify this. First, the basic operation performed in the MR environment and its attributes were investigated, and at the same time, the structure of the hand and the attributes of the possible hand movements were identified. Based on the identified properties, it is intended to present a system that can intuitively and efficiently match basic operation properties in the MR environment with subtle operation properties according to the structure/context of the hand.