Recognition of Real-Time Hand Gestures using Mediapipe Holistic Model and LSTM with MLP Architecture

Abstract

This study used computer vision to capture real-time hand gestures using the MediaPipe Holistic Model and LSTM with MLP architecture to bridge the communication gap between the hearing majority and the deaf minority. The structure of LSTM architecture was consist of 84 neurons with a 30, 1662 input vector that will be transmitted to the MLP model. It has five layers with corresponding nodes of 84, 56, 28,14, and 7 and a dropout with rate values of 0.4 in the first, second, and third layers, while 0.5 in fourth layer. The proposed method was trained and validated with 1000 epoch which achieved a 100 percent accuracy rate in the recognition of real-time hand gesture. Additionally, the researchers envisioned to incorporate hand gesture detection into a number of real-world applications for human-computer interaction in the coming years.