Kinematic Motion Retargeting for Contact-Rich Anthropomorphic Manipulations

Hand motion capture data is now relatively easy to obtain, even for complicated grasps; however, this data is of limited use without the ability to retarget it onto the hands of a specific character or robot. The target hand may differ dramatically in geometry, number of degrees of freedom (DOFs), or number of fingers. We present a simple, but effective framework capable of kinematically retargeting human hand-object manipulations from a publicly available dataset to diverse target hands through the exploitation of contact areas. We do so by formulating the retargeting operation as a non-isometric shape matching problem and use a combination of both surface contact and marker data to progressively estimate, refine, and fit the final target hand trajectory using inverse kinematics (IK). Foundational to our framework is the introduction of a novel shape matching process, which we show enables predictable and robust transfer of contact data over full manipulations (pre-grasp, pickup, in-hand re-orientation, and release) while providing an intuitive means for artists to specify correspondences with relatively few inputs. We validate our framework through demonstrations across five different hands and six motions of different objects. We additionally demonstrate a bimanual task, perform stress tests, and compare our method against existing hand retargeting approaches. Finally, we demonstrate our method enabling novel capabilities such as object substitution and the ability to visualize the impact of hand design choices over full trajectories.