Real-time recognition and translation of Kinyarwanda sign language into Kinyarwanda text

Abstract

Despite significant technological advancements, there continues to be a considerable communication gap between individuals with hearing disabilities and the rest of society. This gap is exacerbated by the fact that the development and research of technologies, such as caption glasses, aimed at bridging this divide, primarily focus on sign languages used in countries with prominent tech industries, including European countries and USA. Consequently, there is a lack of resources and attention devoted to sign language recognition and translation systems for languages spoken in Africa. This research addresses this issue by concentrating on twenty-two common gestures in Kinyarwanda sign language. Through extensive exploration and evaluation of various machine learning algorithms, the study identifies the most effective approach for recognizing and translating these gestures. To validate the effectiveness of the developed system, real-world Kinyarwanda sign language video data is utilized for thorough training and testing. The research successfully culminates in the creation of a functional web application capable of accurately recognizing the 22 Kinyarwanda sign language gestures, both in live video feeds and recorded videos. This achievement represents a significant outcome of the research, as it addresses the specific needs of the Kinyarwanda signing community. By providing a reliable and accessible tool for gesture recognition and translation, the research contributes to narrowing the communication gap between individuals with hearing disabilities who use Kinyarwanda sign language and the wider society.