ViWatch

Wearable devices like smartwatches and smart wristbands have gained substantial popularity in recent years. While enabling a variety of computing applications, such devices are not always convenient to interact with because of the limited size of the touchscreen. A wide variety of approaches have been considered to improve user experiences, ranging from using customized RF sensors, to multiple sensors in smartwatches. These solutions have limitations related to the characteristics of their technology. We propose ViWatch (Vibration Watch), which harnesses vibrations with an IMU sensor on commodity smartwatches to enable fine-grained finger interactions. We detect subtle finger vibrations from noise and design a novel adversarial neural network to mitigate human body variations. ViWatch is able to recognize finger typing and writing induced vibrations with accuracy, even when users are in different states of motion or in noisy environments.