Hand Prosthesis: Finger Localization Based on Forearm Ultrasound Imaging

Abstract

With the advancement in mechanical characteristics of prosthetic hands, the need to develop a novel control strategy is crucial. Although surface electromyography (sEMG) is a functional human-machine interface method in various commercial prostheses, it has practical limitations such as a low signal-to-noise ratio. This paper focuses on the forearm ultrasound imaging method to recognize individual finger movement. In contrast to other published research, dedicated to only discriminating hand gestures, we present a method to control hand prostheses by the angles of each finger. By taking ultrasound imaging from a healthy male subject while flexing and extending his finger, and labeling them through attaching a checkerboard to the fingers, the FUMUS (Ferdowsi University UltraSound) images are produced. Due to the ability of convolutional neural network to extract features, we design an end-to-end system for each of four deep convolutional neural networks named Visual Geometry Group Networks (VGG–16 and −19), MobileNet V1 and V2 and used 90% of our dataset to train the networks and validate their performance in recognizing the label of new forearm ultrasound images. Results show approximately 1 degree Mean Absolute Error (MAE) between the labels of the unseen 10% dataset to neural networks and the exact label of them.