Assistive Multimodal Wearable for Open Air Digit Recognition Using Machine Learning

Abstract

To increase access to digital systems for populations suffering from upper limb motor impairment we present an assistive wearable device to capture gestures performed in air. These open air gestures provide an interface for users who are unable to exhibit the fine motor control needed for standardized human computer interfaces utilizing miniature button input such as keyboards and keypads. By capturing the motion performed at the wrist by an accelerometer as well as the muscle activation signatures using surface electromyography, we improve the classification accuracy as compared to using either modality alone. Twelve features were extracted from the multimodal time series data in both the time and frequency domain and used as input to a collection 4 machine learning models for classification, Fine Tree, K-Nearest Neighbor, Support Vector Machine, and Artificial Neural Network. One subject performed the task of writing single digits in free space and after post-processing and feature extraction we achieved a classification accuracy of 96.2% for binary discrimination of digits zero and one using a support vector machine model and an accuracy of 71% when classifying all 10 digits using an artificial neural network. Our findings indicate the feasibility of a wearable multimodal human computer interface to relieve the burden conventional interfaces present to motor impaired users.