Hand Gesture Recognition using Machine Learning

Abstract

Sign language recognition is a growing area of research, with applications ranging from gestural communication to controlling devices using gestures. One of the challenges inherent to sign language recognition is the ability to translate gestures into meaningful information, such as letters, words or even sentences. Machine Learning, which has emerged as a powerful tool for solving a wide variety of complex problems, namely in the field of computer vision, plays a key role, enabling computers to understand and interpret complex gestures. In this paper, we present a Machine Learning model focused on classifying hand gestures that represent the letters of the Latin alphabet. The objective of this work is to create a solution capable of accurately identifying which letter of the Latin alphabet is being represented by a hand gesture in an image. To classify manual gestures was used the Random Forest Machine Learning classification model, which is fed with the vector of features extracted from the region of interest in the image. To implement the proposed approach, a database of RGB images of hand gestures was created. To extract the characteristics of the gestures, was used the MediaPipe open source framework. The solution presents hand gesture classification precisions by class ranging between 98.8% and 74.4%, with an accuracy of 92.3%, that represents an improvement over previous approaches.