ReflexGest: Recognizing Hand Gestures Under VLC-Capable Lamps

Abstract

As a main approach towards touch-free human-computer interaction, hand gesture recognition (HGR) has long been a research focus for both academia and industry. Meanwhile, visible light communication (VLC) has become increasingly popular with VLC-ready commercial products (e.g., Philips lamps) available on the market. These facts provoke us to ask: can we leverage a VLC-ready lamp to realize integrated sensing and communication (ISAC) by conducting both HGR and VLC simultaneously? To this end, we propose ReflexGest as our answer to this question. ReflexGest is implemented upon a table lamp for the sake of practicality; this VLC-ready lamp is equipped with a ring-shaped light-emitting diode (LED) array and a photodiode (PD, for light intensity sensing) originally aiming for up/down-link VLCs. Demanding hand gestures to be performed between the lamp and a table surface, ReflexGest exploits the variation of the reflection and their unique correlation with the corresponding hand gestures to achieve HGR. In particular, ReflexGest first handles the limited sensing ability of the PD by enhancing the LED lamp and thus diversifying the light emission patterns. Moreover, ReflexGest combats the reflection interference from varying table surfaces via an adversarial learning technique to distill only the features relevant to hand gestures. Our extensive evaluations demonstrate that ReflexGest is able to deliver accurate HGR under realistic VLC traffic.