Gesture Recognition Based Virtual Mouse and Keyboard

Abstract

Nowadays computer vision has reached its pinnacle, where a computer can identify its owner using a simple program of image processing. In this stage of development, people are using this vision in many aspects of day to day life, like Face Recognition, Color detection, Automatic car, etc. In this project, computer vision is used in creating an Optical mouse and keyboard using hand gestures. The camera of the computer will read the image of different gestures performed by a person's hand and according to the movement of the gestures the Mouse or the cursor of the computer will move, even perform right and left clicks using different gestures. Similarly, the keyboard functions may be used with some different gestures, like using one finger gesture for alphabet select and four-figure gesture to swipe left and right. It will act as a virtual mouse and keyboard with no wire or external devices. The only hardware aspect of the project is a web-cam and the coding is done on python using Anaconda platform. Here the Convex hull defects are first generated and then using the defect calculations an algorithm is generated and mapping the mouse and keyboard functions with the defects. Mapping a couple of them with the mouse and keyboard, the computer will understand the gesture shown by the user and act accordingly.