Automatic Digit Hand Sign Detection With Hand Landmark

Abstract

Sign language is challenging to understand and needs a lot of practice before it can be mastered. With the growth of the deaf and the hard-of-hearing community, researchers are trying to find an effective way to understand sign language. This study will utilize hand landmarks to detect digits in sign language. Three models trained with different features will be used to compare their accuracy. The first model will be trained using the hand images only, the second model will be trained using the hand image and the hand landmarks, and the third model will be trained using the hand landmarks only. Mediapipe will be used to extract the hand landmark features, which is one of the features used by the model. The study results show that the first and second models have better training and testing accuracy than the third. However, the third model is superior when evaluated using the validation dataset with 85% accuracy, compared to 23.30% and 41.70% for the first and second models.