Robust dexterous hand control strategy cascading bare hand pose estimation and joint jitter suppression

Abstract

Vision-based dexterous hand control via human hand intuition has great potential in improving control naturalness and immersion, which further achieves better dexterity and generalization. However, challenges still exist in robust control, which is affected by environmental issues including estimation fluctuations and human hand physiological tremor. Hand pose estimation suffers from self-occlusions and self-similarities, and problem exists in balancing stability and hysteresis when suppressing jitters. In this paper, we develop a novel dexterous hand control strategy cascading bare hand pose estimation and joint jitter suppression to enhance controlling robustness. The bare hand pose estimation network utilizes CNNs, ASCS-RL and a biologic-awared refinement module. CNNs extract hand pose features, ASCS-RL obtains accurate hand joint locations. A biological-awared refinement module considering joint movement coupling is novelly modeled and proposed to better refine global hand pose. Meanwhile, joint jitters are reanalyzed, which consist of physiological tremor and error fluctuation. A zero-delay low pass filter with threshold is then introduced to suppress joint jitters. Ablation studies validate the effectiveness of the proposed estimation modules. Best accuracy on ICVL is shown in comparative experiments with recent works to the best of our knowledge, with state-of-the-art accuracy also achieved on other two datasets. Dexterous hand control experiment is finally carried out, where joint jitters are effectively suppressed via the suppression algorithm, and robust dexterous hand control is achieved using the proposed strategy performing static gestures and dexterous object interactions.