Skeleton Guided Conflict-Free Hand Gesture Recognition for Robot Control

The skeleton analysis introduced by Kinect has been an efficient way for interaction. Skeleton analysis interaction is more intuitive and aligns better with human natural behaviors compared with traditional approaches. However, skeleton analysis often has the problem of producing conflict gesture identifications if two interaction movements are similar. Additionally, it always mistakenly recognizes some unconscious or intentional body movements as positive gestures. To this end, we proposed a new interaction method enhanced by both vision algorithms and deep learning. An improved residual neural network is employed to recognize gestures which are then used for distinguishing similar body movements. A combined human-computer interaction scheme is proposed which includes three main components: (a) a hand shape segmentation approach enhanced by skin color detection and skeleton joint tracking, (b) the deep learning augmented detection for changes of gestures and (c) a deep learning-based gesture command recognition for robot control. Experiments are conducted using the proposed method for robot interaction. The results demonstrate that unconscious body movements can be accurately identified. Similar body movements can also be distinguished robustly. The proposed method can run in real-time with competitive performance.