Attention-Based 2-D Hand Keypoints Localization

Abstract

Hand keypoint localization is used extensively in human–computer interaction, but accurate localization is challenging due to closeness between the fingers and the keypoints, occlusion, varied hand poses, complex backgrounds, and extreme lighting conditions. Despite much research, challenges persist. Therefore, we propose an encoder–decoder architecture aided by a novel attention module to precisely localize hand keypoints. The attention module captures keypoint-relevant features at two different scales that encompass local and global characteristics. Further, the loss function teaches the model to remove spurious detected keypoints in the initial learning phase. The proposed architecture outputs precise keypoint locations, as indicated by the qualitative and quantitative results. Evaluation of two benchmark RGB image datasets, comprising all the challenges encountered in keypoint localization, resulted in endpoint errors as low as 2.78 and 1.85 pixels and 98.50% and 99.77% correct keypoints, respectively. This shows the proposed model's effectiveness and ability to overcome challenges.