WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices

Abstract

This paper demonstrates that it is possible to leverage WiFi signals from commodity mobile devices to enable hands-free drawing in the air. While prior solutions require the user to hold a wireless transmitter, or require custom wireless hardware, or can only determine a pre-defined set of hand gestures, this paper introduces WiDraw, the first hand motion tracking system using commodity WiFi cards, and without any user wearables. WiDraw harnesses the Angle-of-Arrival values of incoming wireless signals at the mobile device to track the user's hand trajectory. We utilize the intuition that whenever the user's hand occludes a signal coming from a certain direction, the signal strength of the angle representing the same direction will experience a drop. Our software prototype using commodity wireless cards can track the user's hand with a median error lower than 5 cm. We use WiDraw to implement an in-air handwriting application that allows the user to draw letters, words, and sentences, and achieves a mean word recognition accuracy of 91%.