Person-specific hand reconstruction for natural HCI

Abstract

Seamless interaction between human beings and computer generated 3D objects is an essential task for mixed reality technologies. With the availability of modern wearable tracking systems, it is now possible to interact virtual objects with hand directly. However, without a precise reconstruction of hand geometry, it is difficult to achieve realistic and natural interaction between hand and virtual objects. Today, it is not only necessary to recognize the motion of hand, more importantly, but also necessary to precisely reconstruct the hand geometry, so that a sensible interaction between an actual hand and virtual objects can become possible. In this paper, a technique for natural interaction between hand and virtual objects in virtual environment is proposed. In our method, the 3D hand geometry is implicitly represented as a kind of distance function based on finger joint positions from position tracker and the motion model of the hand is reconstructed according to the data of data glove. The experiment results show that our technique has a much better performance due to the fact that it is much convenient to perform object collision detection and shape matching tasks.