Demo: Finger and Hand Gesture Recognition using Smartwatch

There has been a sharp increase in the popularity of smartwatches in last 2-3 years. Apart from the fitness applications, the smartwatch provides rich graphical interface to users that has enabled applications like email, messaging and navigation using the smartwatch. Since most current smartwatches come equipped with accelerometer and gyroscope sensors, they provide a unique opportunity for gesture recognition. It is expected that user’s arm movements can be identified using the smartwatch easily, however it is not clear how much of user’s hand and finger gestures can be recognized. For example, when user performs a hand gesture such as volume up by rotating hand right, the amount of motion registered with the smartwatch is likely to be very small. Even worse, when the user performs a finger gesture such as zoom-in or zoom-out using fingers and thumb, the movement recorded at the wrist area can be even smaller than hand gestures. If the hand and finger gestures can be recognized using smartwatch, a plethora of applications can be enabled using gesture recognition. For example, user wearing a smartwatch can remotely control nearby television, computer or smartphone, or user can write different characters on a surface to input the text to the smartwatch. In this work, we will demonstrate the feasibility of finger and hand gesture recognition using a smartwatch. In our recent work [3], we showed that the motion energy recorded in the wrist while doing finger or hand gestures is enough to uniquely identify the gestures. We have identified that different tendons passing through the wrist create a unique signature of wrist movement while doing different gestures. In our implementation, we use various features [2] of motion energy, posture and motion shape to learn and recognize different gestures in real-time. Our gesture recognition sys-