Vision-based human–machine interface for a robotic exoskeleton glove designed for patients with brachial plexus injuries

Abstract

This paper presents a novel vision-based human–machine interface (HMI) incorporated into an exoskeleton glove tailored for patients with brachial plexus injuries. Addressing the challenges posed by the loss of hand muscle control in individuals affected by these injuries, a fully automated exoskeleton glove function akin to a robotic gripper is used to prevent muscle atrophy through targeted hand muscle exercises. The proposed vision-based HMI is designed for a fully automated exoskeleton glove and incorporates computer vision techniques for the automatic identification of the target object, estimating its material and size, allowing the precise application of the required force to the target object. This novel approach enables users to efficiently grasp unknown objects with a significantly reduced failure rate. The vision-based method exhibits a grasp success rate of 87.5%, surpassing the baseline slip-grasp method’s 71.9%. These results underscore the effectiveness of our vision-based HMI in enhancing the grasp functionality of the exoskeleton glove.