Design of Smart Shopping Wall Using Hand Gesture and Facial Image Recognition

Abstract

Rapid technological progress has changed the way of people shopping. Online shopping allows consumers to directly buy products or services over the Internet using a web browser. In addition, mobile commerce makes the online sales transactions happen in anytime and everywhere using wireless electronic devices such as mobile phones or laptops. Mobile shopping in QR code virtual stores is a kind of special shopping experience. Using a camera phone with QR code reader installed, a customer can buy the items displayed on the media by flashing their camera phones on the items and the items would be delivered to them through credit card payment. Many QR code shopping walls were created in Mass Rapid Transit stations, malls or public places in big cities recently. In this study, we try to construct a smart shopping wall to enhance the fun in QR code shopping experiences. An integrated system was designed to allow customers controlling the showing page of product Ads on the electronic (TV) displays using skeleton-based hand gesture recognition. The system also can estimate the customer's gender and age via facial image recognition using deep learning algorithms. Microsoft Kinect depth sensor was applied in our system and it provided a good sensor to catch facial images and skeleton information for the purpose of recognition. Experimental results show that the gender and age classification can achieve high recognition accuracy. Finally, an experimental system is completed based on the proposed framework.