A pilot study on hand posture recognition from wrist-worn camera for human machine interaction

Abstract

Hand gestures have been shown to be an efficient way for human-machine interaction. Existing approaches usually utilize ambient or head/chest-mounted cameras to capture hand images. This paper presents a new way to capture hand gestures using the wrist-worn camera. The wrist-worn device is designed as a watch with an integrated camera that is much easier and comfortable to wear in daily life context. We then collect a dataset of ten hand postures using the designed prototype by ten subjects. In addition, we deploy state-of-the-art lite CNN models (YOLO family, Single Shot Detector-SSD) as posture detectors and classifiers. Experimental results show that with limited camera angles, the postures are highly distinctive and easily discriminated with the highest performance of 98.85% and 97.40% in terms of precision and recall, which motivates a wide range of applications and new research directions for human-machine interaction, wearables, the Internet of Things (IoT) and so on.