Hand Tracking Based Human-Computer Interaction Teaching System

Abstract

In the modern era of computers, various new technologies have been arising. One such thing is a touchless application that is used or controlled aerially with hand gestures and movements. Augmented reality and virtual reality have come into use which is controlled by gesture controls. Applications that work with gesture controls have started targeting all kinds of users. Python libraries like MediaPipe and OpenCV are used in hand-tracking, palm detection and object detection. Our work aims in developing a virtual painter that helps young children to draw simple images and shapes of varying sizes. The tool recognizes the hand with hand and palm detector models of MediaPipe and capture the modes for selection and drawing using OpenCV library. In the covid pandemic where children are stuck at home and everything has become online, this tool helps them in practicing simple shapes virtually and also makes it interesting for them. The system is tested by drawing aerially with hands and using selection/drawing modes. It worked well with less time latency due to the inbuilt SSD algorithm used in MediaPipe.