Vision-Based Hand Tracking System Development for Non-Face-to-Face Interaction

Abstract

Human-computer interaction (HCI) focuses on the interaction between humans and computers and it exists ubiquitously in our daily lives, especially in post COVID era where non-face-to-face interaction is common. Since HCI usually uses a physical controller such as a mouse or a keyboard, it hinders National User Interface, giving a middle ground between the user and the computer. This paper presents a vision-based hand tracking system development for non-face-to-face interaction, which aims to improve HCI by being able to track the hand which will act as the pen and functioning as a reusable writing surface for creating texts, drawings, and such as well as removing or erasing using the user’s hand as the pen, and utilizing Open Computer Vision Library (OpenCV) and Mediapipe. Using the computer’s camera the hand will be tracked as the pen for creating basic drawings and handwriting. The vision-based board where the user can draw on and the pen or marker will be the user’s hand. The results indicate that this system is accurate enough to be a feasible application for handwriting ad basic drawings.