A Local Structure-aware 3D Hand Pose Estimation Method for Egocentric Videos

Abstract

In this paper, we propose a method to estimate 3D hand pose from first-person videos. First, in order to remove the effect of background, we build a hand detection module based on YOLO network then apply this module in RGB images. Then, depth information of detected hand regions is employed to construct point clouds. Finally, a local structure aware model named SplitPointnet which consists of six PointNet++ models is proposed to simultaneously estimate joints in five fingers and the thumb region. Experimental results obtained on a large dataset of egocentric vision FPHAB have shown that the proposed method results better hand pose estimations than the state-of-the-art methods with the average error is 66.26 mm.