SimplyMime: A Dynamic Gesture Recognition and Authentication System for Smart Remote Control

Abstract

The widespread use of consumer electronics in today’s society highlights the ever-evolving landscape of technology. With the constant influx of new devices into our households, the accumulation of multiple infrared remote controls, required for their operation, causes not only wasteful energy consumption and resource depletion but also a disordered user environment. To tackle these issues, we present SimplyMime, an innovative system that aims to eliminate the need for multiple remote controls in the realm of consumer electronics, while providing users with an intuitive control experience. SimplyMime uses a dynamic hand gesture recognition framework that seamlessly integrates artificial intelligence with human-computer interaction, allowing users to easily interact with a wide range of electronic devices. The keypoint model used for gesture identification provides a flexible system that can be easily adapted to recognize a variety of hand gestures, even complex ones. In addition, SimplyMime introduces a novel Siamese-based hand palmprint authentication system that acts as the security module for our work and ensures that only authorized individuals can control the devices. The system’s hand detection is enhanced by a customized single-shot multibox detector (SSD) algorithm, which narrows its anchor boxes and uses a feature pyramid network (FPN) to identify hands across different feature maps, serving as a resource-efficient model. Extensive testing on numerous benchmark datasets has proven the effectiveness of our proposed methodology in detecting and recognizing gestures within motion streams, achieving impressive levels of accuracy.