Hand Gesture Recognition System in the Complex Background for Edge Computing Devices

Abstract

Hand gesture recognition offers a wide range of contactless applications. Some applications include designing a sign language recognition system for communicating with people having disabilities, health-care, automobiles, security, etc. For deaf and hard-of-hearing people, sign language recognition is a game-changer and has been studied for years. Unfortunately, each study has its limitations and cannot be used commercially. Some inves-tigations have shown that detecting sign language is possible, but commercialization is prohibitively expensive. This paper investi-gates a robust system to implement hand gesture recognition in a complex background. To test the hand gesture recognition system design, we developed an American sign language recognition for the letters A-J in a complex environment. The MediaPipe Hands framework, used in the developed system, helps successfully detect the hand landmark positions. The machine learning techniques are built on top of the obtained hand landmark positions. The developed system achieves 98.1% accuracy in gesture recognition with an inference time of around 50 ms. Subsequently, the system is successfully ported to Raspberry Pi 4 and NVIDIA's Jetson AGX Xavier and tested.