Tracking of Wrist and Hand Kinematics With Ultra Low Power Wearable A-Mode Ultrasound

Abstract

Ultrasound-based Hand Gesture Recognition has gained significant attention in recent years. While static gesture recognition has been extensively explored, only a few works have tackled the task of movement regression for real-time tracking, despite its importance for the development of natural and smooth interaction strategies. In this paper, we demonstrate the regression of 3 hand-wrist Degrees of Freedom (DoFs) using a lightweight, A-mode-based, truly wearable US armband featuring four transducers and WULPUS, an ultra-low-power acquisition device. We collect US data, synchronized with an optical motion capture system to establish a ground truth, from 5 subjects. We achieve state-of-the-art performance with an average root-mean-squared-error (RMSE) of $7.32^{\circ}$ $\pm$ $1.97^{\circ}$ and mean-absolute-error (MAE) of $5.31^{\circ}$ $\pm$ $1.42^{\circ}$. Additionally, we demonstrate, for the first time, robustness with respect to transducer repositioning between acquisition sessions, achieving an average RMSE value of $11.11^{\circ}$ $\pm$ $4.14^{\circ}$ and a MAE of $8.46^{\circ}$ $\pm$ $3.58^{\circ}$. Finally, we deploy our pipeline on a real-time low-power microcontroller, showcasing the first instance of multi-DoF regression based on A-mode US data on an embedded device, with a power consumption lower than $30 \mathrm{mW}$ and end-to-end latency of $\approx$ $80 \mathrm{ms}$.