Learning Co-occurrence Features Across Spatial and Temporal Domains for Hand Gesture Recognition

Abstract

Hand gesture is the most natural modality for human-machine interaction and its recognition can be considered one of the most complicated and interesting challenges for computer vision community. In recent years, there has been a noticeable advancement in the field of machine learning and computer vision. However, providing a hand gesture recognition system robust enough to work in real-time applications remains challenging. Dynamic hand gestures can be seen as variations in shape or movement during hand motion and often both together. To tackle these challenges, we propose a dynamic hand gesture recognition approach based on hand skeletal sequences. In particular, we introduce a simple but effective deep network architecture to deal with Spatio-temporal co-occurrence features computed on 3D coordinates of hand joints along the gesture sequence. Experimental results show that our approach outperforms state-of-the-art methods on two public datasets, First Person Hand Action and SHREC’2017, with an efficient time computational model compared to most existing approaches.