A robust and efficient automated product recognition system based on scenario design methodology and computer vision techniques

Automated product recognition systems should minimise the online product verification time while keeping product recognition robust to human interventions, including misplaced products, rotated products and empty shelves. A predefined template of each product is generated offline, and scale and location invariant binary local features are employed for recognition. Three types of Sampling Bounding Mask (SBM) are defined and combined with RGB/LUV colour histograms, and matched with a planar level of a shelf by using the Earth Mover’s Distance, thereby reducing the computation time by 50%. Eight comprehensive product views are used to define five product scenarios on a shelf. The SBMs can be used to accurately distinguish the scenarios in a region at a level of shelf. By identifying four scenarios where a shopkeeper does not need to take any action to immediately reallocate misplaced products on shelves, the computation time is reduced by 80%. Six invariant keypoints and feature computation methods are compared for the recognition of product shelf items in different scenarios. The proposed approach speeds up the process by 200% to 300% for product matching. By integrating SBM view detection, the accuracy of product recognition in the front-view with rotations ranges from 90% to 95% using six different feature descriptors.