A Gesture Control System based on Augmented Reality and Eye Tracking Technologies for a 3D-Printed Prosthetic Arm

Abstract

This paper introduces a gesture control system that integrates augmented reality (AR) and eye-tracking technologies for the operation of a 3D-printed prosthetic arm. Within this framework, users wear AR smart glasses equipped with eye-tracking and near-eye display modules. The system recognizes the user's gaze on a variety of everyday gesture AR representations (including open palm, grip, pinch, and point) shown on the near-eye display module. Subsequently, it commands the 3D-printed prosthetic arm to emulate the observed gestures. This approach enhances the synergy between the prosthetic and the natural arm, offering individuals with a single-arm amputation a more organic dual-handed experience in their daily activities.