Palm Reading: Using Palm Deformation for Fingers and Thumb Pose Estimation

Abstract

Hand pose estimation is recognised as being one of the most challenging topics in the field of human pose estimation. Accurate estimation and tracking of multi degree of freedom hand joints can be beneficial to many research areas such as robotic tele-manipulation, motion patterns, robotic hand design and, more generally, human computer/robot interaction. Current solutions to hand tracking are unsatisfactory due to numerous simplifications used in modeling of the hand kinematics and noise-prone hand and finger position sensing methods. In this paper, we propose alternative hand pose sensing approach that includes detecting palm shape in order to more accurately estimate joint angles of middle and index fingers and thumb. We use Inertia Measurement Unit (IMU) sensors on the palm to detect forming of palm arches in different fingers and thumbs’ poses. Principal component analysis as well as Dynamic Neural Networks are utilized to create three different models for fingers and thumb poses, while Polaris optical motion capture system is used as a ground truth. Validating through the unused data shows that using the palm shape as an additional parameter in hand tracking can estimate the hand digit joint angles with the average error of under 4.1%.