Sign Language Hand Gesture Recognition Method based on Machine Learning

Abstract

The dynamic development of computer technology and means of communication and the improvement of network technology have led to an increase in the role of information as a major resource in society. People with hearing impairments, like everyone else, need to benefit from all areas where ICT is applied. Gestures are the only way for people with hearing and speech disabilities to communicate. Automatic recognition of gestures to facilitate communication with gestures is a topical issue, both scientifically and practically. The study provides information on static and dynamic gestures, various sensor technologies used in the collection of gesture data have been researched. The advantages and disadvantages of image-based and non-image-based technologies are analysed. A machine learning method based on neural networks has been developed for high-precision identification of gestures. High results were obtained when testing the developed method on a database open to scientific research. Thus, the method was able to recognize the letters of the dactyl alphabet with an accuracy of 0.95, 0.92, 0.95, 0.94 on the indicators of accuracy, precision, recall, F1-score, respectively.