Hand Landmark Distance Based Sign Language Recognition using MediaPipe

Abstract

The deaf and hard-of-hearing community uses sign language for communication and interaction with the external world. Sign language recognition has been an active area of research for many years, and there has been progress in both sensor-based and vision-based methods. Sensor-based methods, such as those that use gloves or other wearable devices, have historically been more accurate, but vision-based methods are becoming more prevalent due to their cost-effectiveness. The study aimed to recognize sign language words using hand pictures captured by a web camera. The mediapipe hands method was used to estimate hand landmarks, and features were generated from the distances between the landmarks. Support Vector Machine (SVM) classifiers were used for character and words classification. The study used its own dataset and it compared different scaling factors, including the distances from positions 0 to 17, 5 to 17, and 0 to 12, to determine which one worked best. The best results were found using the palm size distance (o–9). The proposed approach is economically feasible and computationally simple, requiring no specialized equipment.