Shoppers Interaction Classification Based on An Improved DenseNet Model Using RGB-D Data

Abstract

This study aims to present a deep learning approach utilizing transfer learning and an RGB-D dataset termed HADA (Hands dataset) acquired by a depth sensor from a top-view configuration capable of monitoring customers and classifying their interaction in intelligent retail settings. With the intention of developing an automated RGB-D approach for video analysis, we provide an innovative, intelligent technology that can comprehend customer behavior, in particular their interactions with items on the shelves. The camera system identifies the presence of humans and classifies their interactions with products accurately. Through the RGB and depth frames, the system determines consumer interactions with shelf objects and identifies if a product is picked up, taken and subsequently returned, or if there is no touch at all. Our approach obtained good accuracy, precision, and recall, demonstrating the efficiency of the proposed model, and testing findings have proved that its performance in real-world conditions is adequate.